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EXECUTIVE SUMMARY 


Randomised trial evaluation of the ln:tuition programme 


Key findings for the primary school trial 

• The primary outcome was resistance skills (confidence to manage 
peer pressure) in 10 and 1 1 year-olds. There was no evidence of any 
impact on this primary outcome. 

• There was an indication of an effect of the intervention on increased 
knowledge (a secondary outcome). On average, primary pupils in 
the intervention group had slightly better knowledge about alcohol 
and its effects than those in the control group, although the results 
were not significant at the 0.05 level (p=0.07). 

• There was no evidence of impact on other secondary outcomes. 

• In terms of programme fidelity, of 40 schools randomised into the 
intervention group, only 15 were known to have delivered at least 
some of the intervention lessons. 

Key findings for the secondary school trial 

• The primary outcome was the proportion of students aged 12-13 that 
were drinking frequently. Overall, there was no significant effect on 
frequency of drinking. 

• In the intervention group, males were more likely and females were 
less likely to be frequent drinkers compared to their counterparts in 
the control group at follow up. However, there is insufficient evidence 
to assert this was a genuine effect of the intervention. 

• There was no evidence of impact on any secondary outcomes. 

• Of 28 schools randomised into the intervention group, only five were 
known to have delivered at least some of the intervention lessons; 
only two secondary schools delivered all or most of the lessons. 

Key findings from the process evaluation 

• Perceived impacts of Induition on pupils included: increased 
knowledge and awareness of alcohol; development of strategies 
and skills to cope with potential social and emotional situations; and 
a change in projected future drinking. 

• Teachers were positive about the programme content and teaching 
approaches but adapted the programme to take account of the 
time available and the needs/context of the school. 
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Background 

This summary reports the findings from two cluster-randomised trials of 
Drinkaware's school-based ln:tuition life skills and alcohol education 
intervention: one trial of the programme for 10-11 year olds in primary schools, 
and another for 12-13 year olds in secondary schools. The trials have been 
carried out by the National Foundation for Educational Research (NFER), 
funded and overseen by Alcohol Research UK using a grant provided by 
Drinkaware. 

Although there has been a downward trend in consumption of alcohol 
amongst 11-15 year olds in England over recent years (Fuller, 2014), two-fifths 
(39 per cent) of young people in this age group have still drunk a whole 
alcoholic drink at least once; nine per cent of those have done so in the last 
week and five per cent usually drink alcohol once a week. The Government’s 
Department of Education (DfE) recognises that effective drug and alcohol 
education is essential to tackling the problem of drug and alcohol misuse. 
Reviews of school-based alcohol misuse prevention programmes, including 
those undertaken by Foxcroft and Tsertsvadze (201 1 ), Cairns et al. (201 1 ) and 
Martin et al. (201 3) have concluded that the evidence base for effective 
alcohol education programmes has been mixed. 

The ln:tuition intervention 

The development of the intervention was informed by evidence-based life 
skills programmes such as ‘Unplugged’ (see Faggiano et al., 2010). Elowever, 
there are differences between the programmes and the measures of impact 
that mean the results are not directly comparable. Alcohol is the main theme 
of the intervention, but its focus is wider and includes, for example, attitudes 
and behaviour, the influence of peers, communication skills and assertive 
behaviour. The main aim of the programme is to delay the age of first 
alcoholic drink. 

The primary and secondary programmes each consist of 12 lessons (a period 
of 40 minutes is recommended for each lesson). Elome learning tasks are 
included to encourage discussion with parents/carers. To access the 
programme, a user needs to register on the website. A helpdesk is available 
via the site. Lesson content is different for primary and secondary schools 
and, therefore, two separate trials have been implemented. A feasibility 
study was carried out in 34 schools across the UK (Barksfield and EHull, 2012). 
The trials were commissioned following this feasibility study. 

Aims of the trials 

The overarching aim of both trials was to evaluate the process and impact of 
implementing ln:tuition in schools. The primary outcomes (those of greatest 
importance) and secondary outcomes (possible additional effects of the 
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programme) were linked to the main aims of the intervention and are 
illustrated below. 
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Methods 

Schools were randomly allocated to receive the ln:tuition intervention or to a 
‘business-as-usual’ control. The trials compared any change over time 
between groups by carrying out a self-report questionnaire survey of students 
at two time points across two school years: before the intervention was 
implemented (June-September 2013) and after (June-July 2014). 

The definitive analysis for the trial was an ‘intention-to-treat’ analysis, which 
includes all students who completed a questionnaire at both time points, 
regardless of the extent of ln:tuition delivery (all intervention schools are 
treated the same). This represents how delivery might occur in a real-world 
scenario. To assess any differential impact by pupil characteristics, we 
conducted sub-group analysis in relation to primary outcomes using 
interaction terms. In addition, ‘on-treatment’ analysis was carried out, which 
accounts for differences in programme delivery. 
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A process evaluation, involving interviews with staff and pupils in nine case- 
study schools and a teacher survey, was carried out to explore programme 
implementation. 


Sampling, recruitment and randomisation 

For the primary school trial the aim was to recruit 70 schools in England 
containing Years 5 and 6. In total, 79 primary schools were recruited (after 
randomisation, 40 intervention schools and 39 control schools). 

For the secondary school trial the aim was to recruit 80 schools in England 
containing Years 7 and 8. In total, 55 secondary schools were recruited (after 
randomisation, 28 intervention schools and 27 control schools). 

Initially, schools were randomly selected from a sampling frame of 33 local 
authorities (LAs). It was then necessary to draw two top-up samples of new 
LAs, leading to a total of 55 LAs and 808 primary schools and 1513 secondary 
schools to achieve the obtained response. Schools that did not agree to 
participate were asked for feedback on the reasons why, although the 
majority said they were ‘unable to help’ and did not give specific reasons 
relating to concerns about the trial or intervention. Difficulties with recruitment 
could have been due to Intuition’s focus on life-skills and alcohol education, 
which are likely to be given less of a priority in schools compared with ‘core’ 
subjects, such as mathematics, English or science (due to timetable and 
curriculum pressures). 

Schools in the control group were prevented from registering on the site 
during the course of the trial, but were given access at the end. 


Results from the primary school trial 

Of the 79 primary schools randomised (into 40 intervention schools and 39 
control schools), there was some attrition before follow-up. A total of 24 
intervention schools and 31 control schools completed both baseline and 
follow-up surveys. This yielded data from 723 intervention and 1019 control 
pupils for the final analysis. Comparisons of their characteristics suggested 
that the final intervention and control groups of primary pupils can be 
regarded as equivalent. 

Of the 40 primary schools randomised into the intervention group, only 15 
were known (based on information provided) to have delivered at least 
some of the intervention lessons. Little information was received about why 
some schools had agreed to participate and then either officially withdrew or 
went on to not deliver lessons. The issues of attrition and programme fidelity 
are likely to have an impact on the measurable effects of the intervention. 

There is no evidence that the ln:tuition programme improves the primary 
outcome of resistance skills (confidence to manage peer pressure in 10 and 
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1 1 year-olds). No significant differences were evident between the 
intervention and control groups and there was no effect on any sub-groups. 

There was an indication of an effect of the intervention on the knowledge of 
primary pupils. The results show that, on average, those in the intervention 
group had slightly better knowledge about alcohol and its effects than those 
in the control group, although the results were not significant at the 0.05 level 
(p=0.07). 

There was no significant difference between the groups in terms of their self- 
reported decision-making skills. Pupils in the intervention group were found to 
be no more likely to understand social norms relating to alcohol (that most 
people will not have a whole alcoholic drink before they are age 1 6) than 
those in the control group 


Results from the secondary school trial 

Of fhe 55 schools randomised (into 28 intervention schools and 27 control 
schools), there was attrition prior to follow-up, particularly for the intervention 
group. A total of 12 intervention schools and 15 control schools completed 
both baseline and follow-up surveys. This yielded data from 586 intervention 
and 814 control pupils for the final analysis. A comparison of fheir baseline 
characteristics yielded imbalance in two variables, neither of which were 
significant predictors of the primary outcome. 

Of fhe 28 secondary schools randomised info the intervention group, only five 
were known fo have delivered at least some of the intervention lessons; only 
two delivered all or most of the lessons. Again, little information was received 
about why some schools had agreed to participate and then either officially 
withdrew or went on to not deliver lessons. As for the primary school trial, the 
issues of attrition and programme fidelity are likely to have an impact on the 
measurable effects of the intervention; these issues were most prevalent 
among secondary schools. 

There was no evidence that participation in the Induition programme had an 
impact on the primary outcome relating to the proportion of students overall 
aged 1 2-1 3 who were drinking frequently. However, males were more likely 
and females were less likely to be frequent drinkers compared to their 
counterparts in the control group at follow up. There is insufficient evidence 
though to assert this was a genuine effect of the intervention. 

Although the proportion of pupils that had had an alcoholic drink by the time 
of the follow up survey was slightly higher in the intervention group than the 
control group, this difference was not statistically significant. Nor was there 
any significant difference between the intervention and control groups in 
relation to the other secondary outcomes. 
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Results from the process evaluation 

Overall, most case-study teachers reported a range of perceived impacts on 
pupils, including increased knowledge and awareness, development of 
strategies and skills and modified behaviour. However, they felt that they 
would have achieved the same impact using existing provision. 

Teachers were positive about the content and teaching approaches but 
adapted the programme to take into account the time available, the 
needs/context of fhe school, contenf covered in other lessons and pupils of 
different abilities in their class. Suggestions for improvements included: 

> reducing the duration and content of the programme 

> providing different formats to deliver the resources such as slides and 
more visual resources 

> greater differentiation of content for pupils with lower levels of literacy 

> pupils would like more opportunities for discussion and more pupil led 
activities. 


Limitations of the trials 

There were a number of limitations to these trials which could have had an 
impact on results: 


• the level of measurement attrition, particularly in secondary schools 

• the nature of the data obtained to conduct analysis on programme 
fidelity 

• the programme materials being made available to schools later than 
expected, which could have impacted on fidelity 

• the involvement of NFER staff in reminding schools to register for the 
intervention, which would not happen in the real world 

• schools included in the process evaluation were those willing to take 
part (and thus are likely to be those most engaged with the 
programme). 

Conclusions 

To conclude, recruitment difficulties were faced, which was followed by 
school attrition (particularly in secondary schools) which could have had an 
impact on results. There was little evidence of positive - or indeed negative - 
impact of the programme. Pupils in schools which delivered the intervention 
did no better - or worse - than those in schools doing their normal Personal, 
Social, and Health Education (PSHE) curriculum. Evidence from the fidelity 
analysis and process evaluation indicates that not all teachers in the 
intervention group delivered the programme in its entirety - indeed some did 


Alcohol Research UK Final Report 


6 


not deliver it at all. Moreover, the extent to which Induition was delivered in 
schools did not impact on this result. Although teachers saw some value in 
the individual elements of the programme, for the programme to have a 
positive impact in its entirety it needs fundamental revision. 
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1. Introduction 


This report summarises the findings from two cluster-randomised trials of 
Drinkaware's school-based ln:tuition programme: one trial of the programme 
for 1 0-1 1 year olds in primary schools, and another for 12-13 year olds in 
secondary schools. The two trials were registered as follows: 

http://www.controlled-trials.com/ISRCTN862241 91 
http://www.controlled-trials.com/ISRCTN7 1 3729 1 3 


The trials have been carried out by the National Foundation for Educational 
Research (NFER), funded and overseen by Alcohol Research UK using a grant 
provided by Drinkaware. During the trials, schools were randomly allocated 
to receive Imtuition or to a ‘business-as-usual’ control. The trials compared 
any change over time for intervention and control groups by carrying out a 
self-report questionnaire survey of pupils at two time points across two school 
years: June-September 201 3 and June-July 2014. 


1.1 Policy background 

Although there has been a downward trend in consumption of alcohol 
amongst 11-15 year olds in England over recent years (Fuller, 2014), there 
remains cause for concern. Two-fifths (39 per cent) of young people in this 
age group have drunk a whole alcoholic drink at least once; nine per cent of 
those have done so in the last week and five per cent usually drink alcohol 
once a week. Alcohol consumption increases with age amongst this 
population; six per cent of 1 1 year olds have ever had an alcoholic drink, 
which increases to 72 per cent by age 1 5 (a fifth of 1 5 year olds have had an 
alcoholic drink in the last week). 

The Chief Medical Officer for England's official guidance on alcohol aimed 
specifically at young people (Donaldson, 2009) recommends that the best 
option is for children to remain alcohol free up to the age of 1 5; these 
statistics show that for a considerable proportion of young people aged 11-15 
this is not the case. The 2012 Alcohol Strategy (F1M Government, 2012) 
includes a commitment to sustain the reduction in the number of 11-15 year 
olds drinking alcohol. 

There are health and societal risks associated with alcohol consumption. 
Annual statistics for young people’s substance misuse interventions 2012-13 
have recently been published by Public Health England 1 (2013), with 4,704 
young people under 18 (24 per cent of the total) having approached 
specialist services with a concern centred around alcohol. In England, the 


1 Public Health England is the national agency tor protecting and improving the nation's 
health and wellbeing and tackling health inequalities. 


Alcohol Research UK Final Report 


8 


Health and Social Care Information Centre (2014) statistics on the costs and ill 
health associated with alcohol consumption showed that, in 2012-13, there 
were an estimated 1,008,850 hospital admissions related to alcohol 
consumption where an alcohol-related disease, injury or condition was the 
primary reason for hospital admission or a secondary diagnosis. Moreover, in 
201 3, there were 1 83,81 0 items prescribed (in a primary care setting or NHS 
hospital) for the treatment of alcohol dependency and dispensed in the 
community. The cost of these prescription items in 201 3 was £3.1 3 million. One 
of Public Health England’s priorities is ‘helping people to live longer by 
reducing preventable deaths from conditions such as heart disease, stroke, 
cancer and liver disease' (Hoskins, 201 3), to which excessive alcohol 
consumption can contribute. 

The Government's Department for Education (DfE), which takes the policy 
lead for young people and alcohol, recognises that effective drug and 
alcohol education is essential to tackling the problem of drug and alcohol 
misuse. In March 2013, the DfE published outcomes of its review of Personal, 
Social, and Health Education (PSHE) (DfE, 2013), which emphasises the 
expectation for schools to use their PSHE programme to equip pupils with an 
understanding of risk and the knowledge and skills necessary to make safe 
and informed decisions. Yet PSHE remains a non-compulsory subject in 
England. Moreover, the official body for inspecting schools in England, 

Ofsted, identified weaknesses in PSHE relating to understanding of the 
physical and social damage associated with alcohol misuse and 
recommended that schools should ensure appropriate learning about these 
issues (Ofsted, 2013). 

Reviews of school-based alcohol misuse prevention programmes, including 
those undertaken by Foxcroft and Tsertsvadze (201 1 ), Cairns et al. (201 1 ) and 
Martin et al. (2013) have concluded that the evidence base for effective 
alcohol education programmes has been mixed. 

The Martin et al. (2013) review was carried out by NFER on behalf of Alcohol 
Research UK to give context to the trials of ln:tuition reported here. They 
concluded that there have been difficulties in judging impact of school- 
based alcohol misuse prevention programmes due to the challenge of 
generalising about effective programme ingredients and issues related to 
programme fidelity (if fidelity has not been investigated through a thorough 
process evaluation, it is difficult to ascertain whether outcomes are a result of 
the effectiveness of the programme, or of the way in which it has been 
implemented). Their review found substantial evidence relating to the positive 
effects of school-based alcohol education and life-skills programmes on 
pupils' alcohol related-knowledge but variable and inconclusive findings in 
relation to pupils' attitudes towards drinking and their decision-making skills. 
They found a degree of evidence of the effectiveness of alcohol education 
and life-skills programmes in reducing the frequency of alcohol consumption 
and episodes of drunkenness among school-aged children. They noted, 
however, limitations to several of the research studies cited. 
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1.2 The ln:tuition Intervention 

Drinkaware is an independent alcohol education charity, funded through 
contributions from the alcohol industry. As part of its preventative work, 
Drinkaware has developed the ln:tuition life skills education programme, 
targeted at young people aged 9-1 4. The main aim of the programme is to 
delay the age of first alcoholic drink. The primary and secondary 
programmes each consist of 12 lessons (a period of 40 minutes is 
recommended for each lesson). Alcohol is its theme but the focus is much 
wider and also includes: 

• self-awareness 

• attitudes and behaviour 

• advertising, branding and the media 

• personal choices 

• emotions, communication skills and assertive behaviour 

• the influence of peers 

• goal setting and confidence. 

A full description of the lessons included in the primary and secondary 
programmes is given in Appendix A. Induition uses a ‘life skills approach' to 
help pupils learn and practice some important skills such as communication, 
working as a team and decision making. The programme can be delivered 
through Personal, Social, and Health Education (PSHE) and Citizenship 
teaching but schools can also adopt a cross-curricular approach. Some 
digital tools are included, but each lesson has a paper-based alternative 
available to download. Home learning tasks are included to encourage 
discussion with parents/carers and promote consistent messages to young 
people at home and school. To access the programme, a member of school 
staff needs fo register on the website and is issued with a verification 
password. A helpdesk is available via the site, should users have any queries. 

The programme was been developed by Drinkaware in partnership with ICE 
Creates. The evaluators have had no involvement in the programme design 
and are therefore entirely independent. Furthermore, a policy was instigated 
by Alcohol Research UK (who managed the evaluation) stipulating that 
members of the evaluation team should not communicate with the 
programme developers (either ICE Creates or Drinkaware) unless a 
representative from Alcohol Research UK was present or copied in to emails. 

Although the structure of the programme is consistent for primary and 
secondary schools, lesson content is different. The delivery contexts in primary 
and secondary schools are also likely to vary. Therefore, two separate trials 
have been implemented and the outcomes are reported separately. 
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1.3 Background evidence 

The development of ln:tuition was informed by evidence-based life skills 
programmes such as ‘Unplugged’, a school-based drug prevention 
programme for pupils aged 12-14 years which was trialled and evaluated in 
seven EU countries (see European Drug Addiction Prevention, 2007 and 
Faggiano et a/., 2010). Note, though, that there are differences between the 
programmes, including content (Unplugged covers the broader issues of 
tobacco and illicit drugs) and the fact that the Unplugged programme 
incorporates two and a half days of training for teachers who will deliver it. In 
addition, the outcomes measured in the trial of Unplugged were not directly 
comparable to those being measured in the trial of In :tuition (because the 
primary outcomes were deemed to be different due to programme content) 
and therefore results are not directly comparable. 

In 201 1 -1 2, CSN was commissioned by Alcohol Research UK to carry out a 
feasibility study and process evaluation in 34 schools recruited from across the 
UK (18 primary schools, 3 middle schools and 1 2 secondary schools). The 
study (Barksfield and Hull, 2012) investigated the implementation of the 
programme in the classroom, the extent to which it was implemented as 
intended, its acceptability to staff and pupils, its relevance and 
appropriateness, barriers to implementation and any requirements for 
additional support or training. A number of recommendations were made as 
a result of fhe study, including: 

• taking steps to encourage better course fidelity, which could be 
assisted by reducing the number of digital tools, identifying a variety of 
curriculum models and, where possible, reducing the length of the 
programme 

• promoting Imtuition as a core life skills course rather than an alcohol 
resource based on a life skills approach 

• making any digital tools that remain in the programme more 
accessible 

• encouraging parental engagement with the ‘home learning' activities. 

Following the feasibility study, Drinkaware invited Alcohol Research UK to 
commission trials of an amended programme. Following an open application 
process, NFER was selected by Alcohol Research UK to conduct a 
randomised control trial and second-stage process evaluation of the revised 
programme. Note that NFER was commissioned as the programme revisions 
were being made. This meant that schools agreed to participate before 
seeing the materials (although an explanation of fheir content was given). 
Details of the trial can be found in the following sections. 
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1.4 Trial objectives 

Alcohol Research UK commissioned trials to provide evidence, if any, of 
Imtuition's potential to deliver longer-term impact and behavioural change 
and to ensure that any future programme development was based on a 
thorough understanding of what does and does not work and why that is 
important to programme success. With this is mind, the overarching aim of 
both the primary and secondary school trials was to evaluate the process 
and impact of implementing Imtuition in schools; see Figure 1 . 


Figure 1: Aims of the trials 
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•Evidence of Intuition's potential to deliver 
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(see chapter 2 for a description of the primary 
and secondary outcomes measured) 
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success 


The main hypotheses were: 


• Primary school trial: the use of the Imtuition programme improves 
resistance skills (confidence to manage peer pressure) in 10 and 1 1 
year-olds 

• Secondary school trial: participation in the Imtuition programme has an 
impact on the proportion of pupils aged 12-13 that are drinking 
frequently 


The primary and secondary outcomes measured in order to address these 
and other related hypotheses are listed in Chapter 2, along with a description 
of the trial methods. 


Alcohol Research UK Final Report 


12 


2. Trial and Analysis Method 


The primary and secondary school trials adopted a cluster-randomised 
design, whereby schools were recruited to the trial and then randomised 
either to receive the In.'tuition programme or not - thus becoming 
intervention and control groups respectively (schools were made aware at 
the time of recruitment that they would be randomised into either group) . 
Control schools continued with their normal PSHE/alcohol education 
curriculum. 

Further details of the trial methodology (including eligibility, sampling and 
recruitment and randomisation) can be found in the primary school trial and 
secondary school trial sections that follow (3.1 and 4.1 ). 


2.1 The outcomes 

The primary outcomes (those of greatest importance) and secondary 
outcomes (possible additional effects of the programme) are illustrated in 
Figure 2. The two primary outcomes reflect the main aims of the programme 
at the different phases. The main long-term goal of both primary and 
secondary school programmes was an impact on young people delaying 
the age of their first alcoholic drink and a reduction in regular drinking. For 
primary schools, the more immediate goal (as few pupils are drinking alcohol) 
was better understanding of strategies that can be used to help make 
responsible choices and confidence to manage peer pressure (resistance 
skills; the primary outcome of the trial). At secondary school age, it is more 
possible to measure current drinking behaviours and therefore the primary 
outcome was frequency of drinking. The secondary outcomes reflect the 
other key aims of the programme at both phases. 

The outcomes and sub-group analyses were agreed with stakeholders before 
baseline measurement took place. Outcome measures are explained in 
Appendix B. 
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Figure 2: Outcome measures 
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2.2 The research instrument 

The evaluation instrument was a self-report questionnaire survey of pupils 
administered at two time points across the school years 2012-13 and 2013-14. 
At both time points, the survey was available on paper and online. The 
questionnaire was identical at both time points. The surveys were designed to 
be age-appropriate, including questions assessing pupil characteristics and 
questions relating to the primary and secondary outcomes (see below) and 
other topics covered in the Irduition programme. Where possible, a variety of 
tried and tested questions were included. 2 Some new questions were 
designed and piloted in a local primary school and amended slightly on the 
basis of discussion with pupils about how they had interpreted questions. The 
final instrument consisted of closed questions, where respondents were 
required to select a response or enter a number. The survey was completed 
by schools at different times during a short survey period (June-July 201 3 and 
September 201 3 3 ), rather than simultaneously, in order to give schools 
flexibility with their timetables. At baseline stage, pupils’ names were 
collected from the questionnaires and retained for the administration of the 


2 Including questions used in the annual survey of smoking, drinking and drug use among 
young people (Fuller, 201 3) and the evaluation of the Talk About Alcohol materials (Lynch et 
al„ 2013). 

3 A flag was added to the final analysis to indicate the timing difference in the administration 
of the baseline survey, which could have an impact on outcomes. 
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final survey in summer 201 4. Pupils were reassured that surveys are 
confidential and that their names are not shared outside of the research 
team. 


2.3 Analysis method 

For both the primary and secondary trials, the 2014 and 2013 survey datasets 
were matched so that each pupil had one record with variables for each of 
the outcomes at baseline and follow up (but just single variables for pupil 
characteristics as these remain constant). 


2.3.1 An ‘intention-to-treat’ (ITT) analysis 

Intention-to-treat (ITT) analysis includes all pupils who completed a 
questionnaire at both time points, regardless of treatment attrition (i.e. the 
extent of Imtuition delivery). This means that the fact that some schools might 
not have delivered the programme as intended is ignored in ‘intention-to- 
treat’ analysis and all intervention schools are treated the same. This was the 
definitive analysis for the trial as it represents how delivery might occur in a 
real-world scenario. Furthermore, it avoids bias that might ensue by excluding 
intervention schools that have not delivered the programme. Such schools 
could share certain characteristics that make them less amenable to positive 
outcomes; excluding them could therefore favour the intervention group. 


2.3.2 On-treatment analysis 

This analysis accounts for differences in programme delivery to determine any 
association between ‘programme fidelity’ (how many lessons of Imtuition 
were actually delivered) and the outcomes of interest. 

Due to issues with intervention fidelity during the trial, an ‘on-treatment’ 
analysis of this type is particularly vulnerable to bias i.e. the schools that have 
delivered the programme as intended are likely to share certain 
characteristics that might influence the outcome. The results of this analysis 
cannot therefore be considered causal. During the trial, it became apparent 
through communications with schools that the intervention was not being 
delivered as intended (see later in the report for details) and, particularly in 
secondary schools, was not always being delivered at all. We therefore 
proposed an additional quasi-experimental analysis for secondary schools to 
compare outcomes for pupils who have experienced the intervention with 
similar pupils in the control group who have not. It transpired that only two 
secondary schools delivered all or most of the lessons so this analysis was not 
possible in the end. 

Information on the extent of programme delivery in schools was obtained 
from a number of sources: the programme developers ICE Creates (based on 
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output from the ln:tuition website, which recorded the schools that had 
registered to access the materials and where some users logged that they 
had delivered lessons); responses to a teacher questionnaire which asked 
which lessons had been delivered; and information volunteered by schools 
(i.e. during NFER contact with schools, some revealed that they had 
registered for the programme but had not gone on to deliver lessons). 


2.3.3 Statistical modelling of primary and secondary outcomes 

The most powerful and unbiased way to analyse the data from these trials, 
that contain measures from the same pupils at baseline and follow-up, is 
through a regression model of follow-up score using baseline score (and 
other predictors) as covariates. This is equivalent to ANCOVA and the reason 
for this being the best approach is discussed by Senn (2006). Since schools 
were randomised, rather than pupils, it was necessary to use a multi-level 
model to ensure standard error (and associated confidence intervals and p- 
values) was correctly estimated. Details of the models, including the 
covariates used, are given in Appendix F. For each model, the following 
procedure was adopted: 

• covariates were checked for high correlation and excluded if deemed a 
risk of multicollinearity 

• a ‘base case' model was run using a two-level (pupil and school) multi- 
level model with no covariates. This returned the pupil-level variance for 
effect size calculation. 

• the outcome was then modelled using a two-level (pupil and school) 
multi-level model containing all pre-specified covariates as measured at 
baseline that may have been influential. Significant variables were 
selected using backward selection. 

• the resulting model was then re-run with intervention forced in (if 
necessary); the intervention coefficient was then used in effect size 
calculations. 


2.3.4 Effect sizes 

A standardised effect size has been presented in Appendix F. For the 
continuous outcomes, the effect size has been calculated as the coefficient 
on the intervention group indicator divided by the pupil-level standard 
deviation from a model with no covariates. For the logistic models (where the 
outcome is binary e.g. yes/no) we use the odds ratio as a standardised effect 
size measure. See Appendix C for information on effect sizes. 
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2.3.5 Confidence intervals 

We have estimated a 95 per cent confidence interval alongside the 
standardised effect size to give the precision with which the effect size has 
been estimated. The upper and lower bounds of the confidence interval 
were calculated as the effect size plus/ minus the product of the critical 
value of the normal distribution (~ 1 .96) and the standard error of the group 
indicator coefficient estimated from the multilevel model. 


2.3.6 Missing data 

Measurement attrition (i.e. drop out from the trial between randomisation 
and endpoint measurements, and hence missing endpoint data) is a 
concern in this as in all trials. Missing baseline data is also an issue. Missing 
data can give rise to biased results as the reason for being missing could be 
related to the outcomes measured. Two strategies were employed to 
address this problem: comparisons of the baseline characteristics of analysed 
groups (see Table 3 for characteristics of primary pupils and Table 13 for 
characteristics of secondary pupils) and multiple imputation. Multiple 
imputation is a statistical technique that attempts to repopulate the dataset 
with reasonable values given other observed factors. Results of the multiple 
imputation were compared with those from the raw data alone in view of 
checking whether the original result was free from bias. Though complicated, 
the technique is a well established way to address possible bias that avoids 
the pitfalls of simpler, though commonly used, alternatives. 4 


2.3.7 Sub-group analysis 

To assess any differential impact by pupil characteristics, we conducted sub- 
group analysis as part of the analysis of the primary outcomes only, using 
interactions. In each case, the final primary outcome model was re-run 
containing the sub-group variable and interaction term. The following 
characteristics were considered: 

• gender 

• ethnicity 

• free school meals (self-reported) 

• age in months 

• siblings/eldest sibling 

• religious group 

• parental drinking in the home 


4 Commonly used alternatives include mean imputation, last observation carried forward or 
using a dummy variable in the model to identify missing cases. 
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attitudes to school (a composite measure). 




2.3.8 Family-wise error rate 

Across both trials we have carried out a total of 28 statistical tests, if you 
include primary and secondary outcomes and sub-group analyses (more if 
you include on-freatment analysis). If these tests were all independent then 
the probability of at least one type I error (concluding the intervention has an 
effect when in fact it does not) is over 0.75. Across the secondary trial alone 
we have carried out a total of 15 sfatistical tests and with the same 
assumption, the probability of af least one type I error is just over 0.5. This 
emphasizes the importance of the primary outcome analysis: all other 
analyses are merely exploratory and would require further research to 
attribute a genuine effect. 


2.4 Reporting 

To construct this report, we used a draft version of CONSORT-SPI, a CONSORT 
Extension for social and psychological interventions, obtained from the 
guideline authors (Montgomery et al„ 2013). 


2.5 Ethical review 

The NFER has a well-developed Code of Practice that contains detailed 
ethical protocols which govern all research undertaken by NFER. The trials 
were agreed by NFER’s Code of Practice Committee in May 2013. The 
Committee also approved research instruments. Letters were sent to parents 
enabling them to withdraw their child from fhe trials (see Appendix D). 
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3. The Primary School Trial 


Key findings 

• Primary outcome: resistance skills - there is no evidence that the In :tuition 
programme improves confidence to manage peer pressure in 10 and 1 1 
year-olds. No significant differences were evident between the intervention 
and control groups and there was no effect on any sub-groups. 

• Secondary outcome: knowledge - the results show that pupils in the 
intervention group were slightly more likely to have better knowledge 
about alcohol and its effects than those in the control group, although the 
results were not significant at the 0.05 level (p=0.07). 

• Secondary outcome: decision-making skills - there was no significant 
difference between the groups in terms of their self-reported decision- 
making skills and, therefore, no evidence that Induition has an impact on 
how pupils make decisions about staying safe and minimising risk. 

• Secondary outcome: understanding social norms relating to the proportion 
of young people who drink alcohol - pupils in the intervention group were 
found to be no more likely to understand the fact that most other pupils 
have not had a whole alcoholic drink 

• Fidelity: Of the 40 schools randomised into the intervention group, only 1 5 
were known to have delivered at least some of the intervention lessons 
(note that three schools officially withdraw from the trial and another three 
schools did not participate in the baseline survey). 


3.1 Methodology 
Trial Design 

The trial adopted a cluster-randomised design, whereby primary schools were 
recruited to the trial and then randomised either to receive the Induition 
programme or not - thus becoming intervention and control groups 
respectively (schools were made aware at the time of recruitment that they 
would be randomised into either group). Control schools continued with their 
normal PSHE/alcohol education curriculum. Intervention and control groups 
were equal in size. 

The trial was designed as an ‘effectiveness’ trial; with every attempt to mirror 
how Induition might be delivered for real. Schools were recruited from a large 
national sample and had minimal assistance with intervention delivery. 
However, due to few schools registering on the Induition website, the 
evaluator agreed in the early stages of the trial to start assisting schools with 
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the registration process. The trial thus became an ‘efficacy’ 5 trial as the level 
of assistance that schools received is unlikely to be matched in a real-world 
delivery scenario. 


Eligibility 

Primary schools in England containing Years 5 and 6, in an agreed sampling 
frame of local authorities (see below), were eligible for the trial. This included 
maintained schools, academies, free schools, pupil referral units and 
independent schools. Special schools and schools which had already 
registered to use the Induition programme were excluded. 


Sample size 

A total of 70 schools for randomisation to two groups of 35 was deemed a 
sufficient sample size to detect an effect size of 0.1 5. Lynch et al. (2012) saw a 
quasi-effect size of 0.1 7 in terms of knowledge outcomes after a similar-length 
intervention. Sample size calculations assumed an intra-cluster correlation of 
0.1 (conservative for attitudinal outcomes controlling for baseline), a 
correlation between baseline and follow-up of 0.8, an effect size of 0.1 5, 
power of 0.80 and 25 pupils per school. The average size of a year group at 
primary school in the UK is 36. However, we have based our calculations on a 
school size of 25 since the geographical constraints of the trial may have 
partially restricted the sample to one-form entry schools. 


Sampling and recruitment 

For the primary trial the aim was to recruit 70 schools for randomisation to 
intervention and control groups. If a school signed up, all pupils in Year 5 were 
recruited to the trial. Schools were randomly selected from a sampling frame 
of 33 local authorities (LAs) agreed with ICE Creates (one of the programme 
developers). Authorities were selected in geographical clusters in order to 
retain other areas for future evaluation of Induition. Schools were excluded if 
they were already registered with ICE Creates. The sample was stratified by 
region and percentage of pupils eligible for free school meals. The initial 
recruitment strategy was to draw large samples of schools, predicting that 
recruitment might be difficult (our experience is that research projects 
relating to life-skills and health-related subjects are often given less of a 
priority in schools than those relating to other ‘core’ subjects, such as 
mathematics, English or science). Indeed, to achieve the required response. 


5 

Efficacy trials aim to determine whether an intervention can work under ideal or developer- 
led conditions. Effectiveness trials aim to determine whether an intervention can work at 
scale, usually in a larger number of schools, where assistance with delivery reflects what is 
possible in a roll-out scenario. 
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it was necessary to draw two top-up samples of new LAs and schools, in 
addition to the original sample. These consisted of 14 and 8 LAs respectively. 
This was despite reminder strategies, which included sending letters with 
information about the trial and the intervention, as well as sending faxes, 
emails and a telephone reminding campaign. The final complete sampling 
frame of LAs is presented on a map in Appendix E. In total, 79 schools were 
recruited (see Section 3.2.1 for details). The total number of primary schools 
invited to take part to recruit 79 schools for the primary trial was 808 across 55 
LAs. 

We sought feedback from schools which did not wish to be included in the 
trial, but despite contacting schools on numerous occasions did not always 
receive a response. A total of 94 primary schools gave feedback, but it was 
not always specific (see Table 1 below). 


Table 1 : Reasons for initial non-participation in the primary school trial 


Reason 

Number of Schools 

Non-specific; unable to help 

83 

Lack of time 

4 

Involved in other projects 

3 

Staff changes/shortages 

2 

Did not feel programme content was 
appropriate 

1 

School circumstances 

1 



Total = 94 



Due to the difficulties with recruitment the baseline survey period was 
extended from the summer term 201 3 into September 201 3. These 
circumstances meant that the questionnaires were completed by schools at 
different times during a survey period that included June-July 2013 and 
September 201 3 6 . 

Randomisation 

Randomisation took place in the summer term 2013 during June and July. 
Once recruited, the 79 schools were allocated randomly to intervention or 
control groups, then informed of their group allocation and asked to 
complete the baseline questionnaire. Schools in the control group were 
prevented from registering on the site during the course of the trial, but were 
given access at the end. 


6 A flag was added to the final analysis to indicate the timing difference in the administration of the 
baseline survey, which could have an impact on outcomes. 
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Randomisation was carried out by an NFER statistician using a full syntax audit 
trail within SPSS v21 . Two adjustments were made to the ideal randomisation 
process. Firstly, rather than the complete group of recruited schools being 
randomised together, schools were randomised in five blocks as and when a 
sufficient number had been recruited. Block sizes were 46, 12, 10, 10 and 1 . 
Secondly, schools were aware of their group allocation when they 
completed the baseline questionnaires, as they were simultaneously notified 
of their group and asked to complete questionnaires. Ideally they would be 
blind to group allocation at this stage to minimise bias so this may have 
affected the results or response rates. For example, schools could have opted 
to complete or not complete the baseline questionnaire based on group 
allocation. 

This approach was adopted mainly to facilitate schools' participation in the 
trial. Schools were understandably keen to know which group they had been 
allocated to as soon as possible so that if necessary they could plan for 
delivering the intervention the following term. Due to the number of lessons 
within the ln:tuition programme, timetabling was a particular challenge for 
schools so they needed as much notice as possible. In addition, recruitment 
took place over a period of several weeks over the summer term 2013. 
Communication with schools needed to take place as early as possible due 
to the summer holidays. It was determined that it was more efficient and 
practical to inform schools of their allocation and ask them to complete 
questionnaires simultaneously. 
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3.2 Results for the Primary School Trial 
3.2.1 Participants 

As shown in Figure 3, a total of 79 schools were recruited and randomised into 
40 intervention schools and 39 control schools. 


Figure 3: primary school trial 
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3.2.2 Attrition and exclusions 


As shown in Figure 3, following randomisation, three intervention and two 
control schools formally withdrew prior to the baseline survey, leaving 37 in 
each group (including 37 in the intervention group expected to register to 
deliver the Induition programme). A total of 34 schools in each group then 
completed baseline surveys, yielding data from 1116 intervention pupils and 
1222 control pupils (a total of 2338) in Year 5 (age 9-10). 


3.2.3 Participants in the end-point survey 

A total of 74 schools were included in the sample for the follow up survey - 
these included all schools which were recruited at baseline and sent 
questionnaires (regardless of whether they completed them) and excluded 
the five schools which actively withdrew from the trial at baseline. A further 
three schools actively withdrew from the end-point survey (two intervention 
schools and one control school). The number of schools completing both 
baseline and follow-up surveys was 24 intervention schools and 31 control 
schools. The number of pupils completing both surveys was 727 intervention 
and 1027 control pupils (although they did not all necessarily answer all 
questions) .This is slightly larger than the number included in the primary 
outcome analysis as the latter required pupils to have a full complement of 
data from the relevant variables. 

Table 2 shows the characteristics of the primary schools included in the 
primary outcome analysis compared with the population of LAs from which 
schools were drawn. There were no significant differences between the final 
analysed sample and the population for region and percentage eligible for 
free school meals (FSM). This suggests that within the final sample we have a 
spread of schools that reflects the population from which they were drawn, 
at least in terms of these two variables. 
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Table 2: Primary school profile at follow up survey 



Population 

In sample 

Number 

% 

Number 

% 

Government 
Office Region 

North East 

633 

16 

11 

20 

North 

West/Merseyside 

604 

15 

11 

20 

Yorkshire & The 
Humber 

541 

14 

8 

15 

West Midlands 

449 

11 

5 

9 

South East 

1136 

29 

11 

20 

South West 

625 

16 

8 

15 

Total N= 


3988 


54 


% students 
eligible for FSM 
2010/11 (5 pt 
scale) 

Lowest 20% 

623 

17 

10 

19 

2nd lowest 20% 

755 

21 

13 

25 

Middle 20% 

793 

22 

8 

15 

2nd highest 20% 

814 

22 

11 

21 

Highest 20% 

690 

19 

11 

21 






Total N= 

3675 


53 



Table 3 shows the baseline characteristics of pupils for whom data was 
collected at baseline and follow up. A school-level comparison between the 
intervention and control group means identified any imbalance between the 
groups in terms of the measured baseline characteristics. None of the 
comparisons revealed statistically significant differences, which suggests that 
the final intervention and control groups of primary pupils can be regarded 
as equivalent. Cases with particular characteristics were not more likely to 
have dropped out i.e. we had no biased attrition in the variables measured. 
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Table 3: Baseline characteristics of primary school pupils 7 


Covariate 

Group 

Number of 
Schools 

Number of 
Pupils 

Mean 

P-value 

Resistance skills at 
baseline 

Control 

30 

1019 

2.94 

0.725 

Intervention 

24 

723 

2.83 

Surveyed after July 

Control 

30 

1019 

0 

0.1 1 1 

Intervention 

24 

673 

0.08 

Age at test in 
months 

Control 

30 

1013 

123.66 

0.572 

Intervention 

24 

720 

123.43 

Self reported free 
school meals 

Control 

30 

987 

0.28 

0.169 

Intervention 

24 

697 

0.20 

Boys 

Control 

30 

1019 

0.51 

0.483 

Intervention 

24 

723 

0.49 

Efhnicity non white 

Control 

30 

1019 

0.1 1 

0.395 

Intervention 

24 

723 

0.08 

Ethnicity unknown 

Control 

30 

1019 

0.75 

0.836 

Intervention 

24 

723 

0.08 

Number of siblings 

Control 

30 

978 

2.15 

0.622 

Intervention 

24 

698 

2.25 

Eldest child 

Control 

30 

988 

0.38 

0.758 

Intervention 

24 

705 

0.37 

Never drink in home 

Control 

30 

1019 

0.19 

0.653 

Intervention 

24 

723 

0.17 

Often drink in home 

Control 

30 

1019 

0.1 1 

0.499 

Intervention 

24 

723 

0.10 

Christian 

Control 

30 

1019 

0.32 

0.538 

Intervention 

24 

723 

0.36 

Other religion 

Control 

30 

1019 

0.05 

0.795 

Intervention 

24 

723 

0.06 

Religion unknown 

Control 

30 

1019 

0.14 

0.347 

Intervention 

24 

723 

0.17 

Attitude towards 
school (composite 
variable from Q7) 

Control 

30 

1019 

29.10 

0.605 

Intervention 

24 

723 

29.0 

Independent or 
Academy school 

Control 

30 

1019 

0.13 

0.570 

Intervention 

24 

723 

0.08 

% pupils eligible for 
free school meals 
(2012/13) 

Control 

30 

1008 

19.52 

0.536 

Intervention 

24 

723 

16.83 

KS2 overall 
performance band 
2013 (average 
point score) 

Control 

30 

945 

2.78 

0.372 

Intervention 

24 

683 

3.13 


7 ‘Surveyed after July’ represents a flag that identifies schools that administered their baseline 
survey in the autumn term. It has a mean value of zero for control schools as it so happens 
that no control schools were surveyed late. 
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Primary schools from which data was collected at baseline and follow up 8 
were grouped into one of four categories depending on the extent to which 
they were known to have completed the In :tuition programme. As shown in 
Table 4 below, only 15 of these primary schools delivered at least some of the 
lessons. 


Table 4: Fidelity of primary intervention schools to the ln:tuition programme 



Number of primary schools 


Completed 

baseline 

only 

Completed 
baseline 
and follow 
up 

All or most lessons completed 

8 

8 

6 or fewer lessons completed 

7 

7 

Known to have completed no lessons (e.g. 
withdrew) 

13 

8 

Known to have registered on the ln:tuition 
website but number of lessons completed 
unknown 

6 

1 

Total 

34 

24 


Due to initial low registration rates on the Induition website, the following 
assistance was offered by the evaluator to intervention schools: the sending 
of an introductory guidebook, assistance with registration via telephone 
conversations and directing school staff to the ICE Creates helpline. 
Furthermore, ICE Creates was occasionally prompted to respond to helpline 
calls. It is not possible to say to what extent this external help altered fidelity 
but it meant that the trial represented a slightly exaggerated picture of how 
well schools might deliver the programme in reality. 

Information on why schools did not go on to register or go on to deliver the 
lessons was limited, but indicates that challenges relating to staff shortages 
and/or limited time available on the timetable were the main factors. 

The issues of attrition and programme fidelity outlined above are likely to 
have an impact on the measurable effects of the intervention. 


8 And therefore had pupils eligible for the ITT analysis 
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3.2.1 Results for the primary outcome: Resistance skills (confidence to manage 
peer pressure) 

As outlined in Section 2.1 , the main aim of the ln:tuition programme in primary 
schools is to equip pupils with a better understanding of strategies that can 
help them make responsible choices and have the confidence to manage 
peer pressure (i.e. resistance skills). Therefore, ‘resistance skills' was the primary 
outcome measured for the primary school trial, based on responses to the 
following questions: 

• if my friends told me to break the rules, I would probably do it 

• I do things just to fit in with my friends 

• I often do what my friends do even if I don't think it is right and it is risky 

• I sometimes make risky choices that might be unsafe. 

At follow-up, primary school pupils tended to indicate that they were able to 
act independently of their friends, if they did not agree with what their friends 
were saying or doing (see Figure 4). Only a small minority of pupils in both 
groups (seven per cent) agreed that they would probably break the rules if 
their friends told them to. Furthermore only one in ten in both groups agreed 
that ‘I often do what my friends do, even if I don't think it is right and it is risky’ 
(nine percent of the intervention group and ten per cent of the control 
group). A slightly greater proportion, yet still a minority, agreed that they 
would do things just to fit in with their friends (16 per cent and 1 7 per cent 
respectively). An even greater proportion (yet this remained a minority) 
agreed that they sometimes made choices that might be unsafe (25 per 
cent of pupils in the intervention group compared with 28 per cent of control 
pupils). 


Alcohol Research UK Final Report 


28 


Figure 4: Primary school pupils' resistance skills (follow-up survey) 



From the results above, there did not appear to be a difference between the 
intervention and control groups. This was explored further with additional 
analysis. A score was derived from pupils’ responses to the questions listed 
above (with a possible range from -8 to 8; see Appendix B for more details). 
Table 5 shows the mean scores for pupils across the intervention and control 
groups (for example, the mean score for pupils in intervention schools was 
3.37). 


Table 5: Mean scores for resistance skills 


Outcome 

Group 

Number 

of 

schools 

Number of 
pupils 

Mean 

Resistance skills 
at follow up 

Control 

30 

1019 

3.21 


Intervention 

24 

723 

3.37 


Statistical modelling was then carried out, which accounts for any differences 
between the intervention and control groups. The main analysis was an 
‘intention-to-treat analysis’ (ITT; see Chapter 2 for details) to determine if there 
was a significant difference in resistance skills between the intervention and 
control pupils. Table 6 shows that there was no significant difference in 
resistance skills between pupils in intervention and control groups. Therefore, 
there is no evidence that the Induition programme improves confidence to 
manage peer pressure in 10 and 1 1 year-olds. 
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Table 6: Effect sizes for the ‘resistance skills’ (primary outcome) model and ‘knowledge’ and ‘decision making skills’ 
(secondary outcomes) models (continuous outcome variables, multi-level models) 
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To investigate this finding further, the characteristics of pupils in the two groups at 
follow-up were explored to see if they can still be regarded as equivalent, at least in 
terms of outcomes we have measured. In this case, there was no evidence of 
imbalance between groups in baseline characteristics (see Table 3) and thus 
groups can be regarded as equivalent. 

Additionally, multi-level multiple imputation of missing data (Ml) 9 was attempted 
using variables that were present in the final primary outcome analysis model. 
Before running the Ml model, measures that had been derived from a series of 
questions were classed as missing if any of the constituent questions were missing. 
For the main ITT analysis, they had only been coded missing if all constituent 
questions were missing; those that were missing being coded to the midpoint of the 
scale for the calculation of the measure. This change resulted in a loss of 209 cases 
and the raw coefficient for intervention altered slightly: from 0.1 64 (0.238) 10 to 0.1 46 
(0.236). In order for the Ml model to run, it had to be simplified to single level. After 
Ml, the raw coefficient for intervention changed to 0.236 (0.159). Thus after Ml the 
non-significant result was retained and, providing cases were missing at random 
given covariates, we can conclude that the result stands. Ml was run in MLwiN 
(version 2.27) using macros downloaded from www.missinadata.ora.uk . 

To assess any possible differential impact on resistance skills by pupil characteristics, 
sub-group analysis was carried out on the ITT dataset introducing: gender; ethnicity; 
free school meals (self-reported); age in months; whether a pupil has siblings/is the 
eldest sibling; religious group; parental drinking in the home and attitudes to school 
(a composite measure) using interaction terms (i.e. interaction between the 
intervention and pupil characteristic). No significant effects of the programme on 
sub-groups were found. 

The intention-to-treat analysis does not take into account the extent to which a 
school followed and completed the Induition programme, so additional, ‘on- 
treatment’ analysis was carried out to take into account the fact that adherence to 
the programme, or ‘fidelity’, had varied considerably between schools (also see 
Chapter 5, the process evaluation) . The on-treatment analysis comprised of a multi- 
level model with the fidelity information added as covariates (see Appendix F for 
the output). There was no significant difference between the groups, therefore 
there is no evidence to show that teaching more of the Induition programme had 
an impact on pupils' resistance skills. 

Further ‘on-treatment’ analysis was carried out to look at whether Induition had an 
impact on resistance skills if pupils received the specific lesson on this topic (lesson 
seven, ‘What should I do?') (see Appendix F for the output). Although resistance 
skills and confidence are themes running throughout the programme, lesson seven 
was most relevant and only known to have been taught in eight of the primary 
schools. * 11 There was no significant difference between the resistance skills of pupils 
that had received lesson seven, and those that had not. 


9 Multiple imputation is a statistical technique that attempts to repopulate the dataset with 
reasonable values given other observed factors. 

10 The standard error appears in brackets. 

11 Lesson data was not available for all schools 
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3.2.2 Results for the secondary outcome: knowledge about alcohol and its effects 

Increasing pupils' knowledge about alcohol and its effects was one of the 
objectives of the ln:tuition programme and consequently was one of the secondary 
outcomes of the primary school trial. All pupils, regardless of whether they had ever 
had an alcoholic drink, were asked whether each of the following six statements 
were true or false, which tested their knowledge of alcohol and its effects on the 
body: 

• ‘Alcohol is a drug’ (True) 

• ‘Alcohol affects different parts of the body’ (True) 

• ‘You can buy alcohol in shops and supermarkets from age 16' (False) 

• ‘A person might have trouble standing or walking if they have drunk alcohol' 
(True) 

• ‘If someone drinks often they will get used to it and it won't harm their body' 
(False) 

• ‘Alcohol is usually swallowed and travels around the body in the blood' 

(True). 


As shown in Figure 5, most pupils in both groups knew that a person might have 
trouble standing or walking if they have drunk alcohol (95 per cent of the 
intervention group and 93 per cent of the control group) and that alcohol affects 
different parts of the body (93 per cent intervention and 91 per cent control). More 
than eight out of ten pupils also knew it was false that if someone drinks often they 
will get used to it and it will not harm their body (81 per cent intervention and 82 per 
cent control). Around two-thirds (64 per cent intervention and 61 per cent control) 
were correct in answering that you cannot buy alcohol in shops from age 1 6 
(although around a third thought you could). There was least certainty, particularly 
in the control group, about whether alcohol is a drug (67 per cent of the 
intervention group compared with 54 per cent of the control group knew it is) and 
about the fact that alcohol is usually swallowed and travels around the body in the 
blood (65 per cent intervention and 57 per cent control group pupils answered this 
correctly). These latter two findings suggest a difference in knowledge of these 
issues between the intervention and control groups at follow-up. This was explored 
with more detailed analysis. 
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Figure 5: Primary school pupils' knowledge of the health effects of alcohol (follow up survey) 
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To investigate any difference in overall knowledge between the groups, each pupil 
was awarded a score between zero and six. The mean score at the school level for 
pupils included in the final model for each group is shown in Table 7, which shows a 
higher mean score for pupils in the intervention group. 


Table 7: Mean scores for knowledge 


Outcome 

Group 

Number 

of 

schools 

Number of 
pupils 

Mean 

Knowledge score 
at follow up 

Control 

30 

1022 

4.37 


Intervention 

24 

726 

4.64 


Multilevel modelling was then conducted to compare knowledge between the 
groups, controlling for any measurable differences between them. The results, 
shown in Table 6, show that pupils in the intervention group had on average better 
knowledge about alcohol than those in the control group, with an effect size of 
0.1 7 (see Appendix C for an explanation of effect sizes) . This result was, however, 
not significant at the p=0.05 threshold (as shown in Table 6). The model intervention 
coefficient was 0.1 97 which means that the intervention group scored on average 
nearly 0.2 points higher than the control group on a scale of 0-6 points. Therefore, 
there is some evidence to suggest that the ln:tuition programme improves 
knowledge about the health effects of alcohol, compared to existing provision. 
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3.2.3 Results for the secondary outcome: decision-making skills 

Improving decision-making skills is one of the main aims of the ln:tuition programme, 
and was therefore one of the secondary outcomes of the primary school trial. 
Responses to the following survey questions were used to explore decision-making: 

• I like to know a lot about something before I do it 

• before I do something, I think about what will happen 

• I tell my friends if I disagree with what they say or do 

• I make choices that help me to stay safe 

• I am able to say no if I don't want to take risks and do something that is 
unsafe. 

At follow-up, more than eight out of ten pupils in both groups reported that they 
made choices that help them to stay safe (86 per cent of the intervention group 
and 89 per cent of the control group) and that they are able to say no if they do 
not want to take risk and do something unsafe (86 per cent of both groups). Most 
liked to know a lot about something before making a decision (85 per cent 
intervention and 86 per cent control). Slightly fewer, yet still around three quarters in 
both groups, felt able to tell their friends if they disagreed with what they said or did 
(73 per cent of the intervention group and 75 per cent of control pupils) and 
reported that they think about what will happen before they do something (74 per 
cent intervention and 73 per cent control). These findings suggest there was little 
difference between the self-reported decision-making skills of pupils in both groups 
at follow-up (see Figure 6). 

Figure 6: Primary school pupils' attitudes towards making decisions (follow up survey) 
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To explore this further, a score was derived from pupils' responses to questions listed 
above, with a possible range from -10 to 10 (see Appendix B for details). Table 8 
shows the average scores for pupils in the intervention and control groups. 

Table 8: Mean scores for decision-making skills 


Outcome 

Group 

Number 

of 

schools 

Number of 
pupils 

Mean 

Decision-making 
skills at follow up 

Control 

30 

1021 

5.84 


Intervention 

24 

727 

5.89 


Multi-level modelling showed that there was no significant difference between the 
groups (Table 6) and therefore no evidence that the ln:tuition programme has an 
impact upon how pupils make decisions about staying safe and minimising risk. 


3.2.4 Results for the secondary outcome: understanding of social norms relating to the 
proportion of young people who drink alcohol 

Improving pupils' understanding of the social norm that most young people will not 
have had a whole alcoholic drink before they are age 1 6, 12 is one of the main aims 
of Induition and hence one of the secondary outcomes of the trial. Table 9 shows 
that the majority of pupils in both groups did not understand this social norm. On 
average, only 24 per cent of pupils in the intervention group, and 28 per cent in the 
control group, understood that most young people would not have a whole 
alcoholic drink before age 1 6. 

Table 9: Mean proportion of pupils answering the social norms question correctly 


Outcome 

Group 

Number 
of schools 

Number of 
pupils 

% giving 
correct answer 

Social norms at 
follow up 

Control 

29 

996 

28 


Intervention 

24 

712 

24 


The results from statistical multi-level modelling (see Table 10) show that pupils in the 
intervention group are no more likely than those in the control group to understand 
the social norm that most people will not have an alcoholic drink before they are 
1 6. Therefore, there was no evidence to suggest that the Induition programme 
improves pupils' understanding of social norms relating to alcohol consumption of 
teenagers. 


12 Two-fifths (39 per cent) of young people aged 11-15 have drunk a whole alcoholic drink at least 
once (Fuller, 201 4) 
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Table 10:Odds ratio for the ‘social norms’ secondary outcome model (binary outcome variable, 
logistic regression) 


Outcome 

description 

Outcome 

measure 

Odds 

Ratio 

95% 

confidence 

interval 

(lower) 

95% 

confidence 

interval 

(upper) 

p-value 

Number of 
intervention 
pupils In 
model 

Number of 
control 
pupils in 
model 

Number of 
intervention 
schools In 
model 

Number of 
control 
schools in 
model 

Secondary 

Understanding 
of social 
norms 

0.83 

0.66 

1.04 

0.12 

712 

996 

24 

29 


Summary 

Overall there is no evidence that the Induition programme has an impact on the 
primary outcome of interest, resistance skills, for primary school pupils. There were 
no significant effects on the secondary outcomes, and although primary pupils in 
the intervention group had on average slightly better knowledge about alcohol 
and its effects than those in the control group, this was not significant either. 
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4. The Secondary School Trial 


Key findings: 

• Primary outcome: frequency of drinking - there was no evidence that 
participation in the ln:tuition programme had an impact on the proportion 
of pupils aged 12-13 who were drinking frequently. 

• Secondary outcome: onset of drinking Although the proportion of pupils 
that had had an alcoholic drink by the time of the follow up survey was 
slightly higher in the intervention group than the control group, statistical 
modeling showed that this difference was not significant. There was no 
evidence of an impact of the programme on delaying onset of drinking. 

• Secondary outcomes: There was no significant difference between the 
intervention and control groups in terms of pupils’: 

o knowledge of alcohol and its effects on health 

o resistance skills (or confidence to manage peer pressure) 

o decision making skills 

o understanding of social norms relating to the proportion of young 
people who drink alcohol (that not all young people under 1 6 will 
have had a whole alcoholic drink) 

• Fidelity: Of the 28 schools randomised into the intervention group, only five 
were known to have delivered at least some of the intervention lessons 
(note that of the 28, three schools officially withdrew from the trial and four 
schools did not participate in the baseline survey or were excluded for 
incorrect administration). 


4.1 Methodology 
Trial Design 

The trial adopted a cluster-randomised design, whereby secondary schools 
were recruited to the trial and then randomised either to receive the Induition 
programme or not - thus becoming intervention and control groups 
respectively (schools were made aware at the time of recruitment that they 
would be randomised into either group). Control schools continued with their 
normal PSHE/alcohol education curriculum. Intervention and control groups 
were equal in size. 

The trial was designed as an ‘effectiveness’ trial; with every attempt to mirror 
how Induition might be delivered for real. Schools were recruited from a large 
national sample and initially had minimal assistance with intervention 
delivery. However, due to few schools registering on the Induition website, the 
evaluator agreed in the early stages of the trial to start assisting schools with 
the registration process. The trial thus became an ‘efficacy’ trial as the level 
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of assistance that schools received is unlikely to be matched in a real-world 
delivery scenario. 


Eligibility 

Secondary schools in England containing Years 7 and 8, in an agreed 
sampling frame of local authorities (see below) were eligible for the trial. This 
included maintained schools, academies, free schools, pupil referral units 
and independent schools. Special schools and schools which had already 
registered to use the Imtuition programme were excluded. 


Sample Size 

In a survey of drinking behaviour in Year 8 within targeted schools, Lynch et 
al. (201 2) showed that 1 1 % were drinking once a month or more. To detect a 
reduction in the prevalence of frequent drinking (once a month or more) 
from 1 0% to 7.5% would require the recruitment of 80 secondary schools for 
randomisation to intervention and control i.e. 40 in each group. This assumes 
a power of 0.80 and an achieved sample of 50 pupils (two classes) per 
school. From the results of models of drinking frequency containing baseline 
data, we did not anticipate an appreciable design effect but the study was 
adequately powered to detect a prevalence change from 10% to 7% with 
roh=0.01 and 1 0% to 6.5% with roh=0.02. 


Sampling and recruitment 

For the secondary trial the aim was to recruit 80 schools with 50 pupils i.e. two 
classes per school. 

Schools were randomly selected from an original sampling frame of 33 local 
authorities (LAs) agreed with ICE Creates (the programme developers). 
Authorities were selected in geographical clusters in order to retain other 
areas for future evaluation of Imtuition. Schools were excluded if they were 
already registered with ICE Creates. As nearly all secondary schools within 
the sampling frame were being drawn in the sample, stratification was 
unnecessary. The initial recruitment strategy was the same as for primary 
schools - to draw large samples of schools, predicting that recruitment might 
be difficult (our experience is that research projects relating to life-skills and 
health-related subjects are often given less of a priority in schools than those 
relating to other ‘core 1 subjects, such as mathematics, English or science). 

As anticipated, recruitment of secondary schools was particularly difficult, 
possibly because they have less curriculum flexibility than primary schools. To 
achieve this response, it was necessary to draw two top-up samples of LAs 
and schools, in addition the original sample. These consisted of 14 and 8 LAs 
respectively. This was despite reminder strategies, which included sending 
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letters with information about the trial and the intervention, as well as sending 
faxes, emails and a telephone reminding campaign. 

The final complete sampling frame of LAs is presented on a map in Appendix 
F. 

During the sign-up process, schools were asked to declare which three or four 
classes 13 they wished to nominate for the intervention. This nomination 
occurred before randomisation to ensure that schools could not choose 
classes on the basis of which group (intervention or control) they had been 
allocated. Schools were trusted to stick to these nominations so they are not 
a complete guarantee that class choice did not change after 
randomisation. 

We sought feedback from schools which did not wish to participate, but 
despite contacting schools on numerous occasions did not always receive a 
response. A total of 106 secondary schools gave feedback (see Table 1 1 
below). 


Table 11: Reasons for non-participation in the secondary school trial 


Reason 

N Schools 

Non-specific; unable to help 

78 

Too short notice (*see below) 

9 

Lack of time 

6 

Involved in other projects 

3 

Did not feel programme content was 
appropriate 

4 

Timetable concerns 

3 

Too few students 

2 

School closing 

1 

Total = 106 



Due to difficulties with recruitment the baseline survey period was extended 
from the summer term 2013 into September 2013. These circumstances meant 
that the questionnaires were completed by schools at different times during a 
survey period that included June-July 2013 and September 201 3 14 . 

Randomisation 

The randomisation process was identical to that adopted for the primary 
school trial, as outlined in Section 3.1 (except block sizes for secondary 
schools were 38, 6, 5, 5 and 1 ) . 


1 3 In order to ensure 50 pupils per school were retained at follow-up. 

14 A flag was added to the final analysis to indicate the timing difference in the 
administration of the baseline survey, which could have an impact on outcomes. 
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4.2 Results for the Secondary School Trial 
4.2.1 Participants 

As shown in Figure 7, a total of 55 schools were recruited and randomised into 
28 intervention schools and 27 control schools. The total number of secondary 
schools invited to take part to recruit 55 schools for the trial was 1513 across 
55 LAs. 


Figure 7: Secondary school trial 
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4.2.2 Attrition and exclusions 

As shown in Figure 7, following randomisation, three intervention and two 
control schools withdrew prior to the baseline survey, leaving 25 in each 
group (including 25 in the intervention group expected to register to deliver 
the Imtuition programme). A total of 21 intervention schools and 20 control 
schools then completed baseline surveys, yielding data from 1729 
intervention pupils and 1 331 control pupils (a total of 3060) in Year 7 (age 1 1 - 
12 ). 


4.2.3 Participants in the end-point survey sample 

A total of 50 schools were included in the sample for the follow up survey - 
these included all schools which were recruited at baseline and sent 
questionnaires (regardless of whether they completed them) and excluded 
the five schools which actively withdrew from the trial. The number of schools 
completing both baseline and follow-up surveys was 12 intervention schools 
and 1 5 control schools. The number of pupils completing both surveys was 
757 intervention and 933 control pupils. This is larger than the number 
included in the primary outcome analysis as the latter required pupils to have 
a full complement of data from the relevant variables. Table 12 below shows 
the characteristics of the secondary schools in the primary outcome analysis 
compared with the population of LAs from which schools were drawn. There 
were no significant differences between the final analysed sample and the 
population for region and percentage FSM. This suggests that within the final 
sample we have a spread of schools that reflects the population from which 
they were drawn, at least in terms of these two variables. 


Table 12: Secondary school profile at follow up survey 




Population 

In sample 



Number 

% 

Number 

% 

Government 

North East 

179 

16 

9 

35 

Office Region 

North 

West/Merseyside 

126 

11 

1 

4 


Yorkshire & The 
Humber 

108 

10 

4 

15 


West Midlands 

131 

12 

3 

12 


South East 

391 

35 

6 

23 


South West 

183 

16 

3 

12 

Total N= 


1118 


26 


% students eligible 

Lowest 20% 

123 

15 

2 

9 

for FSM 2010/11 (5 

2nd lowest 20% 

215 

26 

4 

17 

pt scale) 

Middle 20% 

234 

29 

6 

26 


2nd highest 20% 

163 

20 

9 

39 


Highest 20% 

83 

10 

2 

9 







Total N= 

818 


23 
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Table 13 shows the baseline characteristics of pupils for whom data was 
collected at baseline and follow up. A school-level comparison between the 
intervention and control group means identifies any imbalance between the 
groups in terms of fhe measured baseline characterisfics. There were fwo 
significant differences found between the pupils in the control and 
intervention groups. Pupils in the control group were more likely than 
intervention pupils to be an older sibling, and those in the intervention group 
were more likely than control pupils to describe themselves as belonging to a 
religious group other than Christian. None of the other comparisons revealed 
statistically significant differences, which suggests that the final intervention 
and control groups of secondary school pupils can be regarded as broadly 
equivalent. 

Table 13: Baseline characteristics of secondary school pupils 


Covariate 

Group 

Number of 
Schools 

Number of 
Pupils 

Mean 

P- 

value 

Frequency of drinking 
at baseline 

Control 

15 

814 

0.0518 

0.591 

Intervention 

1 1 

586 

0.0616 

Surveyed after July 

Control 

15 

814 

0 

0.251 

Intervention 

1 1 

586 

0.0909 

Age at test in months 

Control 

15 

811 

147.8046 

0.86 

Intervention 

1 1 

586 

147.8529 

Self reported free 
school meals 

Control 

15 

814 

0.241 1 

0.563 

Intervention 

1 1 

586 

0.2145 

Boys 

Control 

15 

812 

0.4813 

0.921 

Intervention 

1 1 

582 

0.4859 

Ethnicity non white 

Control 

15 

814 

0.21 12 

0.219 

Intervention 

1 1 

586 

0.1 104 

Ethnicity unknown 

Control 

15 

814 

0.0388 

0.859 

Intervention 

1 1 

586 

0.0414 

Eldest child* 

Control 

15 

792 

0.4053 

0.034 

* 

Intervention 

1 1 

570 

0.3276 

Number of siblings 

Control 

15 

794 

2.1763 

0.925 

Intervention 

1 1 

579 

2.1916 

Comprehensive to 
sixteen 

Control 

15 

814 

0.2667 

0.974 

Intervention 

1 1 

586 

0.2727 

Comprehensive to 
eighteen 

Control 

15 

814 

0.2667 

0.279 

Intervention 

1 1 

586 

0.0909 

Grammar school 

Control 

15 

814 

0.1333 

0.223 

Intervention 

1 1 

586 

0 

Rural school 

Control 

14 

747 

0.0714 

0.813 

Intervention 

10 

518 

0.1 

How often does 
anyone you live with 
drink alcohol inside 
your home? (Q26) 

Control 

15 

765 

0.6377 

0.166 

Intervention 

1 1 

545 

0.7249 
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Covariate 

Group 

Number of 
Schools 

Number of 
Pupils 

Mean 

P- 

value 

Number of different 
ethnic categories 
(school) 

Control 

13 

715 

12.5385 

0.684 

Intervention 

9 

438 

13.1 1 1 1 

Attitude towards 
school (composite 
variable from Q7) 

Control 

15 

814 

28.496 

0.279 

Intervention 

1 1 

586 

28.0935 

Christian 

Control 

15 

814 

0.0983 

0.305 

Intervention 

1 1 

586 

0.1379 

Other religion* 

Control 

15 

814 

0.0324 

0.046 

* 

Intervention 

1 1 

586 

0.1981 

Religion unknown 

Control 

15 

814 

0.019 

0.491 

Intervention 

1 1 

586 

0.0134 

Have you ever been 
drunk? 

Control 

15 

814 

0.0774 

0.854 

Intervention 

1 1 

586 

0.0816 

Age when first drank 
an alcoholic drink 
(Q 1 5) 1 5 

Control 

15 

814 

12.9299 

0.399 

Intervention 

1 1 

586 

12.7441 

Have you ever drank 
alcohol without a 
responsible adult 
present? (Q16A) 

Control 

15 

813 

0.0662 

0.165 


Secondary schools from which data was collected at baseline and follow- 
up 16 were grouped into one of four categories depending on the extent to 
which they were known to have completed the ln:tuition programme. As 
shown in Table 14 below, only five of these secondary schools delivered at 
least some of the lessons. 


Table 14: Fidelity of secondary intervention schools to the ln:tuition programme 



Number of secondary 
schools 


Completed 
baseline only 

Completed 
baseline 
and follow 
up 

All or most lessons completed 

2 1 

2 

6 or fewer lessons completed 

3 J 

3 

Known to have completed no lessons (e.g. withdrew) 

13 

5 

Known to have registered on the ln:tuition website but 
number of lessons completed unknown 

3 

1 

Total 

21 

11 


15 In order to boost the number of cases that could go into the models, anyone who said that 
they had never drank alcohol by the time of the follow up survey were assigned an age of 
first drink greater than or equal to 1 4. 

16 And therefore had pupils eligible for the completers (ITT) analysis 
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Due to initial low registration rates on the ln:tuition website, the following 
assistance was offered by the evaluator to intervention schools: the sending 
of an introductory guidebook, assistance with registration via telephone 
conversations and directing school staff to the ICE Creates helpline. 
Furthermore, ICE Creates was occasionally prompted to respond to helpline 
calls. As for the primary school trial, it is not possible to say to what extent this 
external help altered fidelity but it meant that the trial represented a slightly 
exaggerated picture of how well schools might deliver the programme in 
reality. 

Information on why schools did not go on to register or deliver the lessons was 
limited, but indicates that challenges relating to staff shortages and/or limited 
time available on the timetable were the main factors. 

The issues of aftrifion and programme fidelity outlined above are likely to 
have an impact on the measurable effects of the intervention; these issues 
were particularly prevalent for the secondary school trial. 


4.2.4 Results for primary outcome: Frequency of drinking 

One of the main aims of fhe ln:tuition programme in secondary schools is to 
reduce the frequency of drinking among teenagers. Consequently, this was 
agreed as the primary outcome of the trial in secondary schools. The analysis 
of frequent drinking was based on pupils who drank alcohol once a month or 
more (see Appendix B for an explanation of how the measure was 
constructed). 

Table 15 below shows that, on average, around one in ten pupils in the 
intervention and control schools were frequent drinkers at the time of the final 
survey. Looking at the proportion of frequent drinkers out of those who had 
ever had a drink (Figure 8), one in five reported drinking once a month or 
more (21 per cent of intervention group pupils and 20 per cent of control 
group pupils); around one in ten said that they did not drink alcohol any 
more ( 1 0 per cent of intervention group and 1 1 per cent of control group 
pupils). Most pupils only drank alcohol a few times a year, for example on 
special occasions (70 per cent of the intervention pupils who had ever had a 
drink; 69 per cent control). 


Table 15: Percentage of frequent drinkers at follow-up (out of all pupils eligible for the model) 


Outcome 

Group 

Number of 
schools 

Number of 
pupils 

% of pupils at 
follow-up 

Frequency of drinking 
regularly at follow up 

Control 

15 

814 

9 


Intervention 

11 

586 

11 
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Figure 8: Frequency of drinking alcohol among secondary school pupils who had ever had a 
whole alcoholic drink by the time of the follow up survey (n=652) 



Multilevel modelling was then carried out to control for measurable 
differences befween the intervention and control pupils to explore if fhere 
was any difference in fhe proportion who drank frequently at follow-up. The 
main analysis was an intention-to-treat (ITT) analysis (see Section 2.3.1 ). The 
results of the modelling (see Table 1 6) show that there was no significant 
difference between the groups and therefore no evidence that participation 
in the ln:tuition programme had an impact on the proportion of pupils aged 
1 2-1 3 who were drinking frequently. 
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Table 1 6 Odds ratio for the ‘frequency of drinking’ (primary outcome) and ‘onset of drinking’ (secondary outcome) models (binary variable, 
logistic regression) 
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To explore this finding further, the characteristics of pupils in the two groups at 
follow-up were explored to investigate if they can still be regarded as 
equivalent, at least in terms of measures we have. There was some evidence 
of imbalance between analysed groups in baseline characteristics (see 
Section 4.2.3 above). This probably resulted since attrition between baseline 
and follow-up was so extensive. The proportion of pupils who were the eldest 
sibling was significantly higher in the control than the intervention groups (41 
per cent and 33 per cent respectively) and the proportion who were of a 
non-Christian religion was significantly higher in the intervention group as 
compared to control (20 per cent and three per cent respectively). Note that 
neither of fhese variables was significanf in fhe primary oufcome model so 
their imbalance perse is not a problem, however they may be indicative of 
wider imbalance that we could not measure. Due to the large attrition 
between baseline and follow-up, multiple imputation of missing data was not 
possible with the secondary school dataset. 

To assess any differential impact on frequency of drinking by pupil 
characteristics, sub-group analysis was carried out on the ITT dataset 
including: gender; ethnicity; free school meals (self-reported); age in months; 
whether a pupil has siblings/is the eldest sibling; religious group; parental 
drinking in the home and attitudes to school (a composite measure) using 
interaction terms. As summarised above, the ITT analysis showed that overall, 
the ln:tuition programme had no significant effect on frequency of drinking. 
However, a statistically significant finding emerged relating to the interaction 
between the intervention and gender (see Table 1 7) . This suggests that the 
intervention worked differentially for males and females to the extent that 
males were more likely and females were less likely to be frequent drinkers 
compared to their counterparts in the control group at follow-up (Figure 9). 
However, as explained in Section 2.3.8, this analysis is exploratory and would 
require further research to attribute a genuine effect of the intervention. No 
other significant effects of the programme on sub-groups were found. 

Secondary school primary outcome (frequency of drinking) gender sub-group analysis 


Variable name 

Fixed 

Effect 

Standard 

Error 

Degree of 
Freedom 

value 

p-value 

Intercept 

.117 

.514 

1363 

.228 

.820 

Intervention 

-.640 

.368 

22 

-1.738 

.096 

Frequency of drinking at 
baseline 

1.849 

.262 

1363 

7.050 

.000 

Comprehensive school to age 
18 

-1.222 

.456 

22 

-2.682 

.014 

Grammar school 

-1.470 

.622 

22 

-2.363 

.027 

Have you ever been drunk? 

1.134 

.253 

1363 

4.481 

.000 

Age of first alcoholic drink 

-.214 

.035 

1363 

-6.174 

.000 

Male 

.029 

.258 

1363 

.114 

.910 

Male in the intervention group 

.855 

.383 

1363 

2.234 

.026* 


* statistically significant interaction 
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Figure 9: Probabilities of being a frequent drinker: the interaction between the intervention 
and gender for frequent drinking 



ITT analysis does not consider the extent to which the intervention is followed, 
so additional ‘on-treatment’ analysis was carried out to examine whether the 
intervention had a different effect in schools that had completed the 
ln:tuition programme than schools that did not. As described in Chapter 5, 
adherence to the programme, or ‘fidelity’, varied considerably between 
schools. The on-treatment analysis comprised of a logisfic regression with the 
fidelity information added as covariates (see Appendix F for the output). 
There was no significant difference between the groups, therefore there is no 
evidence to show that teaching more of the ln:tuition programme has an 
impact on frequency of drinking in pupils. This means that although it was 
hypothesised that the extent to which the In :tuition programme is delivered 
matters in terms of reducing the frequency with which pupils drink alcohol, 
there is no evidence to support this theory. 

Further ‘on-treatment’ analysis was carried out to look at whether Induition 
had an impact on frequency of drinking if pupils received the specific lessons 
on this topic (lessons two and three, ‘Alcohol, the body and the law') (see 
Appendix F for the output). Lessons two and three were only known 17 to have 
been taught in five of fhe secondary schools. There was no significant 
difference between the frequency of drinking of pupils that had received 
lessons two and three, and those that had not. Note that due to the small 
number of schools involved in this comparison, it is likely that it was 
underpowered i.e. unable to detect a genuine difference should it exist. 


17 Lesson data was not available for all schools 
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4.2.5 Results for secondary outcome: onset of drinking 

Pupils were asked ‘have you ever had an alcoholic drink - a whole drink, 
more than just a sip/taste?’ which required a binary (‘yes/no’) response. 

Multilevel modelling analysis explored the impact of the Induition lessons on 
delaying the onset of drinking - are pupils in the intervention group less likely 
to have started to drink by the follow up survey? 

To add context to the modelling, it should be noted that the proportion of 
pupils that had had at least one whole alcoholic drink 18 by the time of the 
follow up survey (when most pupils were age 12-13) was slightly larger in the 
intervention group (56 per cent) than the control group (48 per cent) (Table 
1 8) . Furthermore, Figure 1 0 shows how the drinking behaviour of pupils who 
had not had an alcoholic drink at baseline varies at follow up. Almost three in 
ten intervention group pupils (29 per cent) who had not drunk at baseline 
had had at least one whole drink of alcohol by the time of the follow up 
survey, compared to just under a quarter of the control group (23 per cent) . 

Table 18: Proportion of secondary school pupils that had had an alcoholic drink by the time 
of the follow up survey” 


Outcome 

Group 

Number 

of 

schools 

Number of 
pupils 

% pupils 

Drinking at follow 
UP 

Control 

15 

835 

48 


Intervention 

11 

605 

56 


18 Including those who had already had an alcoholic drink at baseline 

19 The data reported in this section is based on analysed data only (i.e. the cases that went into the 
final primary outcome model) for consistency with the rest of the chapter. 
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Figure 1 0: Drinking behaviour at follow up of the secondary school pupils who had not had an 
alcoholic drink at the time of the baseline survey 20 



Out of the group of pupils who had already had an alcoholic drink by the 
time of the follow up survey, the mean age at which they did this was similar 
across the two groups: an average age of 10.8 years in the control group 
and an average of 10.5 in the intervention group. 

The results of the multilevel logistic regression (Table 16) revealed that pupils 
who had received the Induition lessons were no more or less likely to ever 
have had an alcoholic drink than those in the control group and, therefore, 
there is no evidence that the Induition programme influences the onset of 
drinking among 12-13 year olds. 


4.2.6 Context: drinking behaviour in 12-13 year olds 

In addition to onset and frequency of drinking (described in Sections 4.2.4 
and 4.2.5 above), secondary school pupils were asked further questions 
about their drinking behaviour 21 . Note that the results in this section have not 
been tested for statistical significance because they were not specified as 
primary or secondary outcomes. 

As shown in Table 1 9, over half of pupils that had drunk a whole alcoholic 
drink said they had never been drunk. However, three in ten pupils that had 


20 Based on analysed data only, i.e. the cases that went into the final primary outcome model 

21 The results reported in this section are based on the whole sample at follow up, and not just the cases 
eligible for the models. 
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drunk alcohol in the intervention group reported that they had been drunk at 
least once (30 per cent), compared to a quarter in the control group (25 per 
cent). A higher proportion of intervention group pupils said they had drunk 
five or more alcoholic drinks on one occasion than control group pupils (30 
per cent and 22 per cent respectively, N=753). Furthermore, almost half of 
drinkers in the intervention group said they had consumed alcohol without a 
responsible adult present, compared to a third of drinkers in the control group 
(48 per cent and 33 per cent respectively, N=838). Overall, this paints a 
picture where pupils in the intervention group drink more heavily than those in 
the control group, and are more likely to be unsupervised, but it must be 
emphasised that these differences have not been tested for significance, 
and it is possible that some of these activities may have taken place prior to 
participating in the ln:tuition programme. 

Table 1 9: The number of times pupils who said they had drunk a whole alcoholic drink 
reported having ever been drunk 



Control 

Intervention 

Never 

53% 

51% 

Once 

15% 

15% 

2-5 times 

7% 

11% 

6-10 times 

2% 

3% 

More than ten times 

1% 

1% 

1 don't know 

23% 

19% 

Total 

100% 

100% 

N= 

390 

440 


Pupils tended to drink in the company of family, with over six in ten of both 
groups reporting that they drink with parents/carers, and around three in ten 
drinking with their siblings or other relatives (see Figure 11). Around one in ten 
said they drank alcohol with friends of their own age. Very few reported 
drinking on their own (three per cent (15 people) in the intervention group, 
two per cent ( 1 0 people) in the control group) . 
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Figure 11: Who do secondary school pupils drink with? (follow up survey; all who ever had a 
drink) 


% pupils 

0 % 10 % 20 % 30 % 40 % 50 % 60 % 70 % 


on my own 
with my girlfriend or boyfriend 
with friends of the same sex as me 


with friends of the opposite sex 
with a group of friends of both sexes 
with my parents or carers 
with my brother, sister or other relatives 
with friends of my own age 
with friends younger than me 
with friends older than me 


with other people 



■ Control 

■ Intervention 


The vast majority of pupils that drank alcohol said they did so for special 
occasions such as birthdays or Christmas (86 per cent in the intervention 
group and 85 per cent in the control group). The next most common reason 
was because they like the taste of alcohol (49 per cent in the intervention 
group and 42 per cent in the control group). Very few pupils said that they 
drank because they felt pressured to drink by their friends (five per cent of the 
intervention group and three per cent of the control group). 


Pupils who had never had a drink of alcohol 

At the point of the follow up survey, 861 pupils said they had not had a whole 
drink of alcohol. Around one in ten of these pupils thought that they would 
have had a whole drink before they turned 16(11 per cent in the intervention 
group and eight per cent in the control group) while about a third were 
unsure (35 per cent and 31 per cent in the intervention and control groups 
respectively). 
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The most common reasons given for not ever having drunk alcohol were: 

• it is bad for your health (intervention 56 per cent, control 62 per cent) 

• I don't feel it is right to drink at my age (intervention 52 per cent, control 
41 per cent) 

• I have learned about the negative aspects about alcohol (intervention 
36 per cent, control 35 per cent) 

• it is against the law to buy alcohol if you’re under 1 8 (intervention 36 
per cent, control 30 per cent) 

• religious reasons (intervention 1 5 per cent, control 28 per cent) . 


4.2.7 Secondary outcome: Knowledge of the health effects of alcohol 

As part of the aim of the Induition programme on improving drinking 
behaviour, the lessons include content intending to improve pupils' 
knowledge of the effects of alcohol, and therefore this formed one of the 
secondary outcomes of the secondary trial. All pupils, regardless of whether 
they had ever had an alcoholic drink, were asked six ‘true or false' questions 
which tested their knowledge of alcohol and its effects on the body: 

• ‘Police can take alcohol from under 18s drinking in a public place 1 
(True) 

• ‘You can buy alcohol in shops and supermarkets from age 1 6’ (False) 

• ‘Alcohol is usually swallowed and travels round the body in the blood' 
(True) 

• ‘If you drink alcohol on an empty stomach the effects are stronger' 
(True) 

• ‘Someone over 1 8 can buy alcohol for me as long as I don’t buy it 
myself ‘ (False) 

• ‘If you stop drinking and switch to soft drinks or coffee then it will speed 
up the rate at which alcohol leaves the body' (False). 


Most pupils knew that the police can take alcohol from under 18s in a public 
place (85 per cent in both the intervention and control groups). Almost three 
quarters of pupils were correct that you cannot buy alcohol from age 1 6 (71 
per cent of the intervention group and 69 per cent of the control). Around six 
out of ten pupils knew that if you drink on an empty stomach the effects are 
stronger (66 per cent of intervention pupils and 60 per cent of control group 
pupils) and that alcohol usually swallowed and travels round the body in the 
blood (64 per cent of intervention pupils and 65 per cent of control pupils). 
However, there were particular gaps in knowledge in relation to the other 
questions. The majority of pupils were wrong in thinking that if you switch to 
soft drinks or coffee it speeds up the rate at which alcohol leaves the body 
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(85 per cent intervention pupils and 83 per cent control group pupils 
answered this incorrectly) and more than halt thought someone over 1 8 
could purchase them alcohol from a shop (61 per cent of the intervention 
group and 58 per cent of the control group answered this incorrectly). 

A knowledge score was calculated for each pupil based on their answers to 
the questions listed above (also see Appendix B). Out of a possible score of 
six, the mean knowledge score was just over three in both the intervention 
and control groups (see Table 20). 


Table 20: Mean knowledge scores at the follow up survey 


Outcome 

Group 

Number 

of 

schools 

Number of 
pupils 

Mean 

Knowledge score 
at follow up 

Control 

15 

829 

3.36 


Intervention 

11 

594 

3.37 


Mean scores out of a possible range of 0 to 6 


The results from statistical modelling also showed that there was no statistically 
significant effect of fhe Induition programme on knowledge of alcohol and its 
effects (Table 21 ). 
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Table 21: Effect sizes for 'knowledge', 'resistance skills', ‘decision making skills’ and ‘social norms' (secondary outcomes) models (continuous 
variables, multilevel models) 


Number of 
control schools 
in model 

LO 

LO 

LO 

LO 

Number of 
intervention 
schools in 
model 

— 

— 

CN 

CN 

Number of 
control 
pupils in 
model 

829 

924 

923 

846 

Number of 
intervention 
pupils in 
model 

594 

671 

754 

691 

P- 

value 

0.691 

LO 

00 

oo 

d 

oo 

oo 

d 

0.399 

95% 

confidence 

interval 

(upper) 

-0.09 

-0.08 

-0.15 

-0.07 

95% 

confidence 

interval 

(lower) 

0.14 

0.09 

0.03 

0.18 

Effect 

size 

0.02 

o 

o 

-0.06 

0.05 

Outcome 

measure 

Knowledge 
of the 
Effects of 
Alcohol 

Resistance 

Skills 

Decision 

Making 

Skills 

Social 

Norms 

Outcome 

description 

Secondary 

Secondary 

Secondary 

Secondary 
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4.2.7 Secondary outcome: Resistance skills (confidence to manage peer pressure) 

Resistance skills, or giving pupils the confidence to manage peer pressure was 
integral to the life skills approach of ln:tuition, so this formed another of the 
secondary outcomes. In order to measure resistance skills, a score was derived from 
pupils' responses to the following questions: 

• if my friends were drinking alcohol I would want to drink as well 

• if my friends told me to break the rules, I would probably do it 

• I do things just to fit in with my friends 

• I often do what my friends do even if I don’t think it is right and might involve 
some risk 

• I sometimes make risky choices that might be unsafe. 

Between six and seven out of ten pupils in both groups disagreed with most of the 
statements above (suggesting they make independent decisions and can manage 
peer pressure), although fewer than half disagreed with the final statement (see 
Figure 1 2) . Nearly a third (32 per cent) of the intervention group and 29 per cent of 
the control group agreed that they sometimes made risky and unsafe choices. 


Figure 12: Proportion of secondary pupils that disagreed to some extent with the resistance skills 
statements (follow up survey) 



■ Control 

■ Intervention 


If my friends told me to 
break the rules, I would 
probably do it 


If my friends were drinking I do things just to fit ii 
alcohol I would want to my friends 

drink as well 


with I often do what my friends I sometimes make risky 
do even if I don't think it is choices that might he unsafe 
right and might involve some 
risk 


A score was derived from pupils’ responses, out of a possible range of -10 to 10, with 
10 representing the highest level of resistance skills. The control group had a mean 
score of 4.03 compared to the intervention group mean of 3.42 (Table 22). 

However, the results from statistical modelling show that there was no statistically 
significant effect of receiving the ln:tuition lessons on resistance skills (Table 21 ) . 
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Table 22: Mean score for resistance skills 


Outcome 

Group 

Number 

of 

schools 

Number of 
pupils 

Mean 

Resistance skills at 
follow up 

Control 

15 

924 

4.03 


Intervention 

11 

671 

3.42 


Mean scores out of a possible range of -10 to 10, with 10 representing the strongest 
resistance skills 


4.2.8 Secondary outcome: Decision-making skills 

As part of the life-skills approach, the ln:tuition programme aims to build pupils’ 
decision making. This formed one of the secondary outcomes of the secondary 
school trial, based on the following questions: 

• I think about what might happen to me before I do something 

• I think about what might happen to other people before I do something 

• I like to understand all the facts before making a decision about something 

• When I make a decision, I often think about the effects it might have. 

Self-reported decision making skills were very similar across the two groups of 
interest, for example three-quarters of the intervention and control groups said that 
they think about what might happen to other people before they do something (74 
per cent and 75 per cent respectively). A similar proportion thought about what 
might happen to them (73 per cent of the control group and 71 per cent of the 
intervention group). Furthermore around two thirds of secondary pupils said that 
they like to understand all the facts before making a decision (66 per cent of the 
control group and 64 per cent of the intervention group agreed or strongly agreed 
with this statement), and that they often think about the effects it might have when 
they make a decision (69 per cent of the control group and 66 per cent of the 
intervention group agreed or strongly agreed with this statement). 

As decision making was one of the secondary outcomes of the trial, a score was 
derived from pupils’ responses to the questions listed above (also see Appendix B). 
The control group had a similar mean decision-making score to the intervention 
group (see Table 23). 
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Table 23: Mean decision making scores of secondary school pupils 


Outcome 

Group 

Number 

of 

schools 

Number of 
pupils 

Mean 

Decision-making 
skills at follow up 

Control 

15 

923 

3.48 


Intervention 

12 

754 

3.27 


Mean scores out of a possible range of -8 to 8 , with 8 representing the strongest decision 
making skills 


The multilevel modelling confirmed that there was no evidence of impact of the 
ln:tuition lessons on decision-making; there was no statistically significant difference 
in average scores between the groups (see Table 21 ). 


4.2.9 Secondary outcome: Understanding of social norms relating to the proportion of 
young people who drink alcohol 

In addition to giving pupils facts and information about alcohol and its effects, 
ln:tuition also aimed to improve pupils' understanding of social norms relating to the 
proportion of young people engaging in drinking behaviours. The survey asked 
secondary school pupils whether the following questions were true or false; 

• ‘Most young people aged 11-15 have never drunk a whole alcoholic drink’ 
(True) 

• ‘Most people aged 1 5-1 6 get drunk regularly' (False) . 

The evidence shows a lack of understanding of social norms relating to alcohol. At 
least eight out of ten pupils answered both questions incorrectly. Only a fifth (20 per 
cent of the intervention group and 18 per cent of the control group) knew that 
most young people aged 11-15 will not have had a whole alcoholic drink, and 
fewer ( 1 7 per cent and 1 6 per cent) were correct in thinking that most 1 5-1 6 year 
olds do not get drunk regularly. 

A score was derived from pupils' responses to the two ‘true or false’ questions, and 
multi-level modelling was carried out on this measure. Pupils in the intervention and 
control groups appeared to have similar levels of understanding of social norms; on 
average the intervention group scored 0.37 out of 2 compared to the average 
control group score of 0.34 (see Table 24). 
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Table 24: Mean score for understanding of social norms 


Outcome 

Group 

Number 
of schools 

Number of 
pupils 

Mean 

Social norms at 
follow up 

Control 

15 

846 

0.34 


Intervention 

12 

691 

0.37 


Mean score out of a possible range of 0 to 2 


The multi-level modelling showed that there was no statistically significant 
difference in the average scores of pupils in the two groups; there was no evidence 
that the Imtuition programme improved pupils’ understanding of social norms 
relating to alcohol consumption of teenagers (see Table 21 above). 


Summary 

Overall there is no evidence that the Imtuition programme has an impact on the 
primary outcome of interest, frequency of drinking of secondary school pupils, and 
there were no significant effects on the secondary outcomes either. This indicates 
that the Imtuition programme is not meeting its stated aims. 
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5. The Process Evaluation 


Key findings 

• Overall, case-study teachers reported a range of perceived impacts of 
Induition on pupils relating to the aims of the programme and the outcomes 
of interest of the trial, including increased knowledge and awareness, 
development of strategies and skills and modified behaviour. However, most 
felt that they would have achieved the same impact using existing PSHE 
provision. 

• Teachers were positive about the content and teaching approaches, but 
adapted the programme to take into account the time available, the 
needs/context of the school, content covered in other lessons and pupils of 
different abilities in their class. 

• Suggestions for improvements included: 

> reducing the duration and content of the programme 

> providing different formats to deliver the resources such as slides and 
more visual resources 

> greater differentiation of content for pupils with lower levels of literacy 

> pupils would like more opportunities for discussion and more pupil led 
activities. 


The main purpose of the process evaluation was to examine: 

• fidelity (whether the programme was delivered as intended) 

• how the programme was implemented 

• parental engagement and use of hometasks 

• effective elements in programme delivery 

• views on programme content 

• challenges and barriers to delivering a successful programme 

• suggestions for improvements. 

These factors are likely to be central to whether the programme has a statistical 
effect on the outcomes of interest explored above. As noted in Chapters 3 and 4, 
there is no evidence of any statistical effects of the programme on the outcomes 
measured. The process evaluation findings could help to explain why this is the 
case, and help to understand elements of the programme which have been 
successful or a particular challenge. 
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5.1 The Process Evaluation Methodology 

This section is based on findings from the case-study visits to schools carried out in 
June and July 201 4. All intervention schools recruited to the trial which then went on 
to register on the ln:tuition website, were invited to take part in the more-in-depth 
qualitative investigation. The aim was to achieve five primary schools and five 
secondary schools with varying background contexts. In reality, given difficulties 
faced with general recruitment to the trial, schools included were those willing to 
take part (and thus are likely to be those most engaged with the programme). This, 
and the fact that only small numbers of schools and individuals were included in this 
qualitative element of the trial (see Table 25), means that the results are illustrative 
and should not be generalised. 


Table 25: Case-study participants 



Number of 
schools 

Number of staff 

Number of pupils 

Primary schools 

6 

6 

46 

Secondary 

schools 

3 

6 

24 

Total 

9 

12 

70 


The findings from the case-study visits supplemented those from a teacher survey, 
to which small numbers of teachers responded (22 teachers across 1 9 primary 
schools and eight teachers across five secondary schools). 


5.2. Results 

5.2.1 Programme fidelity 
Fidelity: were 12 lessons delivered? 

Of the 22 primary school teachers who responded to the teacher survey, only three 
delivered the entire programme of 12 lessons, while 18 did not (one did not 
respond). Of the eight secondary school teachers, only two delivered the entire 
programme 22 . Tables 26 and 27 illustrate which lessons were delivered by teachers 
who responded to the survey. 


22 This information informed the on-treatment analysis described in Chapters 3 and 4. 
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Table 26 : Lessons delivered in primary schools (teacher survey) 


Table 26 Lessons delivered In primary schools (teacher survey) 



Lesson 

Number 

who 

delivered 

lesson 

Of those, 
number 
who felt 40 
minutes 
was about 
right 

If 

delivered 
, number 
rating 
lesson 
very 

good/go 

od*** 

If 

delivered, 
number 
rating 
lesson very 
poor/poor* 
• • 

Lesson 1 : How shall we start 

IS 

12 

10 

1 

Lesson 2: This is me 

17 

15 

13 

- 

Lesson 3: This is me 

15 

11 

11 

1 

Lesson 4: Say it like it is 

13 

11 

10 

- 

Lesson 5: Say it like it is 

12 

12 

8 

2 

Lesson 6: Social norms 

14 

7* 

9 

- 

Lesson 7: What should 1 do? 

10 

8 

6 

1 

Lesson 8: Alcohol and the body 

11 

5 ” 

8 

- 

Lesson 9: Seeing is believing 

7 

3 

7 

- 

Lesson 10: Seeing is believing 

S 

2 

4 

- 

Lesson 1 1 : Looking to the future 

4 

3 

3 

- 

Lesson 1 2: Looking back, looking 
forward 

4 

3 

2 

- 


Total N=22 

• Five felt 40 minutes was too short. ’’Four felt 40 minutes was too short 
’’’Those who delivered the lessons did not olways rote how much they liked them 


Table 27 Lessons delivered in secondary schools (teacher survey) 


Table 27 Lessons delivered in secondary schools (teacher survey) 


Lesson 

Number 

who 

delivered 

lesson 

Of those, 
number 
who felt 40 
minutes 
was about 
right 

If 

delivered 
, number 
rating 
lesson 
very 

good/go 

od 

If 

delivered, 
number 
rating 
lesson very 
poor/poor 

Lesson 1 : How will it start? 

6 

5 

6 

' 

Lesson 2: Alcohol, the body and the 
law 

7 

6 

6 


Lesson 3: Alcohol, the body and the 
law 

6 

6 

6 

- 

Lesson 4: Social norms 

7 

5 

6 

* 

Lesson 5: Say it like it is 

5 

5 

5 


Lesson 6: Say it like it is 

4 

4 

3 

- 

Lesson 7: Standing up, standing out 

3 

3 

3 

- 

Lesson 8: Standing up, standing out 

3 

3 

4 

- 

Lesson 9: Being assertive 

6 

4 

6 

- 

Lesson 1 0: 1 get by with a little help - 

from my friends 

2 

2 

- 

Lesson 1 1 looking to the future 

2 

1 

2 

- 

Lesson 12: Looking back, looking 
forward 

2 

1 

2 

- 


Total N= S 
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Across case-study schools, at the time of the visits in June and July 2014 only two 
primary and two secondary schools had completed the programme in its entirety. 
Several schools (at the time of interview) hoped to complete additional sessions 
prior to the end of term. Of those that had not completed the programme, several 
reported having started the programme and then having run out of time or 
reported that allocated sessions had subsequently been used for other priorities. 

Twelve of the 22 primary teachers who responded to the survey felt that the overall 
length of 1 2 lessons each for 40 minutes was ‘about right'; three said ‘too long’ and 
two said ‘too short’ (the remaining five did not respond). Of the eight secondary 
school teachers who responded, four said fhe programme length was ‘about right', 
while three others who responded said ‘too long'. The tables above indicate that 
the majority of teachers did not complete the entire programme (numbers 
delivering lessons reduced towards the end of the programme). The main reason 
for not completing the programme was lack of curriculum time ( 1 3 primary 
teachers and four secondary teachers). 

Fidelity: were lessons delivered sequentially? 

Tables 26 and 27 (based on data from the teacher surveys) show that lessons were 
not always delivered sequentially; some teachers taught lessons later in the 
programme after not delivering some of the earlier lessons. Therefore, shows 
teachers were selective about which lessons they taught. There was also case-study 
evidence of variability in the extent to which schools delivered ln:tuition as a 
sequential, structured series of 40-minute lessons. About half of the case-study 
schools adapted their delivery approach to suit the needs and circumstances of 
the school. The main reason for this deviation related to the overall length of the 
programme: 

It was too long for us to manage with fhe rest of fhe curriculum we need to 

follow in school (primary) 

[We] had to cut it down to get it into the 6 weeks time frame available -Jan- 

Feb half term (secondary) 


Fidelity: were 40 minutes allocated to each lesson? 

In case-study schools, there was variability in the length of time devoted to 
individual lessons. In about half of the schools, the recommended 40 minutes per 
lesson had been timetabled. Staff in these schools reported that some lessons took 
longer than others, but there was sufficient flexibility to accommodate this. In the 
other schools, including two of the secondary schools, timetables meant that 
around 30 minutes was available for Induition. Comments included: 

I spent 30 minutes on each session - fhe same amount of time as would 
normally be given over to a circle time session - [we] crammed it in a bit 
(primary). 

Because of fhe school fimetable and structure we only have a 30 minute 
session available. But [staff] have adapted the content accordingly 
(secondary). 
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Tables 26 and 27 show that teachers who responded to the surveys did not always 
think 40 minutes for each individual lesson was ‘about right’. Some were considered 
too short, while the case-study evidence suggests variations in views about whether 
schools are able to allocate this amount of time to such lessons. 

Fidelity: were lesson plans used as intended? 

There was evidence that most teachers adapted the content of lessons to suit their 
needs. In case-study schools, the majority of interviewees noted adapting the 
lesson plans to some extent, generally by cutting out some content, largely 
necessitated by time constraints and the amount of material to be covered. In 
other cases, lesson plans were modified to take into account content already 
delivered in other lessons; to focus on particular needs/issues relevant to the school; 
and to tailor aspects of session delivery to the needs/preferences of the class. 
Comments included: 

This was a challenge , there was a lot to fit in the lessons. We didn’t have the 
full 40 minutes as there was only 25 minutes of pastoral time available. So 
that’s one of the main reasons why I have adapted the lessons and made it 
more verbal - saves a lot of time by not getting them to work through the 
worksheets and write it down (secondary). 

I generally followed the plans but often just did verbal feedback sessions 
rather than following the activities and homework tasks specified in the 
programme (primary). 

The majority of interviewees commented on the amount of information the lesson 
plans contained, and whilst recognising their quality and comprehensive nature, 
many suggested they were ‘too wordy' and teachers were faced with ‘information 
overload’ prompting them to modify the lesson plans in order to facilitate session 
delivery: 

I had whot the objectives were, and had the resources, but maybe didn’t do 
it exactly as it was there, because I don’t want to be constantly reading the 
sheets, so you use some of your professional judgement as well, and deliver it 
in a way that is flowing (primary). 

Findings from the teacher survey revealed that five of the 22 primary teachers and 
four of the eight secondary teachers felt the lessons were too detailed (and 
therefore might have adapted them). 

Survey findings also suggested that some teachers did not feel the lessons were 
appropriate for all pupils and, therefore, may have adapted them. For example, six 
teachers in primary schools and three in secondary schools did not think they were 
suitable for all abilities. Eight in primary schools did not think they were appropriate 
for different religions. 

If teachers surveyed were to use the programme again, 1 4 of the 22 in primary 
schools and six of the eight teachers in secondary schools would like to select bits of 
it and adapt it. 
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5.2.2 Programme implementation 

Most teachers who responded to the survey (15 of the 22 in primary schools and 
seven of the eight in secondary schools) taught the lessons individually over a 
period of days/weeks, rather than as an intensive block of lessons grouped 
together. In secondary schools, lessons were most often taught in tutorial time or as 
part of fhe Personal, Social and Healfh Educafion (PSHE) curriculum. 

In case-sfudy schools, the three primary and one secondary school that had 
delivered the programme in its intended format generally ran ln:tuition sessions on a 
weekly basis, either replacing previous PSHE lessons or as part of timefabled 
pastoral sessions. The other case study schools generally implemented a block 
delivery approach in response to timetable constraints and the primary schools also 
largely opted to deliver the programme in the summer term after SATs had been 
completed (note we asked primary schools to deliver the programme in Year 6). As 
a teacher in a primary school said: 

Delivering in a block is often the only way we can do these education pieces. 


5 . 2.3 Parental engagement and use of hometasks 

Only six of the 22 primary teachers who responded to the teacher survey, and four 
of the eight surveyed in secondary schools, introduced the programme to parents. 

If they did, it was most often by letter/in writing, although two primary schools held a 
meeting for parents dedicated to the programme. 

In case-study schools, introductory meetings had not been held in any of the 
schools for the following reasons: 

• No perceived need to introduce ln:tuition to parents: Interviewees suggested 
that parents trusted the school to deliver high-quality, appropriate PHSE-type 
material, so it was not necessary specifically introduce ln:tuition which was 
just seen as another area of curriculum to be delivered. 

• Difficulties in engaging parents in school-life. Both secondary and primary 
school staff noted difficulties in engaging parents with the school and that 
securing attendance at meetings was often problematic. Several staff spoke 
of fheir reluctance to overburden parents with requests to come into school 
and preferred to reserve these requests for ‘more important meetings’ 
(primary). 

• Sensitivity around ln:tuition subject matter. In several schools (primary and 
secondary), staff noted their reservations about raising the profile of the 
ln;tuition programme content amongst parents. 

• Logistics and timing. Several primary school interviewees noted that meetings 
were not held as there had not been sufficient time to organise a meeting 
prior to implementing the programme or that staff were not yet sufficiently 
confident in the delivery of fhe programme, so would be unable fo respond 
to parents' questions. Across the case-study schools, interviewees reported 
very little feedback from parents prior to, or during, the programme's delivery. 
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None of the 22 primary school teachers surveyed sent home all of the homework 
tasks and only seven sent home some of the homework tasks. Among the eight 
secondary school teachers, none sent home all hometasks; three sent home some. 
Six primary school teachers had received negative feedback from parents (the 
survey did not gather details). 

Amongst case-study schools there was variability in the extent to which homework 
tasks were used with interviewees in around half of the six primary schools and two 
of the three secondary schools noting that homework had not been given. In one 
secondary school, this was because ln:tuition was delivered in scheduled 
guidance/support sessions for which homework was not ever given. In the other 
secondary school, individual staff members delivering the programme had opted 
not to give out the homework tasks. Primary school staff noted that they had not set 
homework tasks largely as a result of school-based issues - such as the homework 
timetable having already been established prior to the onset of Induition and the 
existing levels of homework from other curriculum areas. Primary school staff 
contended that they aimed to encourage parents to focus more on supporting 
children with numeracy and literacy, so did not want anything to interfere with this - 
‘it muddied the waters if we then tried to add in PSHE'. 

Several pupils who had not completed hometasks noted that they would have 
been interested in doing them alongside their parents: 

It would have been quite interesting to do it - to do the quiz with my parents 
to find out how much they knew - test what they know about it (secondary 
school pupil). 

Where homework was given, interviewees noted that it was kept ‘fairly low key' to 
try and encourage pupils and their parents to engage, and the tasks relating to 
alcohol was most commonly distributed: 

With oil the pressure on literacy and numeracy, I didn’t push this quite as 
much and made it a bit more informal for the pupils so didn’t push the 
homework aspect (primary) 

Primary staff reported that some parents had engaged with the tasks and some 
completed worksheets had been returned. Pupils were said to have enjoyed the 
tasks, but as a result of time constraints, and low levels of returns, homework was not 
generally integrated into lessons, other than some low-level discussion and verbal 
follow up. A secondary school interviewee noted that the alcohol-related task had 
been particularly well received by pupils who enjoyed working through the sheet 
with their parents. In this case, the task was used as a discussion point in subsequent 
lessons and the staff member suggested that they would like to expand this 
element of the programme in the future: 

They [pupils] liked the fact that parents had homework. They were very 
talkative about what their parents had done, how they and scored. It proved 
to be a big talking point (secondary) . 

At first, mine were quite surprised about the questions that they asked, but just 
went along with it. It made them realise just how things have changed and 
how different society is from then to now and how many more people are 
smoking and drinking at a young age now (secondary pupil). 
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5.2.4 Effective elements in programme delivery 

In case-study schools, primary and secondary school staff highlighted a range of 
factors that facilitated the effective delivery of the In :tuition programme. There 
were no differences evident between the phases. 


Effective lesson plans 

The vast majority of interviewees noted that the lesson plans were instrumental in 
the successful delivery of the programme, although as noted previously, these had 
often been subject to alteration (largely to accommodate time constraints). Lesson 
plans gave structure to the lesson; clearly highlighted the learning objectives; 
provided comprehensive background information; and set out the necessary 
activities to be completed. In one secondary school, this was seen as particularly 
beneficial to the large number of non-teaching staff delivering Induition: 

It was especially good for non-teaching staff .... As a teacher, you can 
navigate through a lesson - but we have a lot of non-teaching staff and the 
lessons plans were especially helpful in explaining to them what they needed 
to do (secondary). 

Where adaptations had been made, they generally involved the introduction of 
extension activities and examples to reflect the school and locality contexts and 
needs of particular pupil cohorts. 


Effective resources and activities 

Most interviewees in case-study schools valued the resources available to support 
the delivery of Induition, commenting on their quality, appropriateness and 
effectiveness in facilitating pupils’ engagement in the sessions. The more visual and 
interactive resources were particularly valued. However, several interviewees had 
modified and adapted resources and the way they were used. This was particularly 
so in relation to worksheets which were not always considered an appropriate or 
effective means of delivery. As a result, the worksheet topics were taken and 
delivered in ways compatible with the school's normal approach in which pupils 
had a greater level of involvement and creativity. 

Both primary and secondary staff valued the pupil-centred interactive and 
dynamic nature of some programme activities whereby pupils spent a lot of the 
session ‘doing’ something and the teachers acted more as facilitators. 

Alongside this, enhanced collective learning was facilitated through the division of 
the class into small groups which encouraged the active participation of all (or 
most) class members in discussions. Most interviewees identified the role play 
sessions and scenarios as being a particular strength of the programme. The 
following quotations illustrate staff and pupils’ thoughts on how Induition activities 
facilitated the programme's successful delivery: 

Mixing the children out of their normal friendship groups - splitting up all the 
confident ones who usually sit with each other. Having mixed ability groups 
helped encourage wider participation of more children (primary). 
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They love role play - they’d do it all day if they could. They get really into it. 
We used the worksheets to help and guide them, but they didn’t really fill 
them all in because they were busy doing the more interactive tasks - they 
learn better that way (secondary). 

Pupils' perspectives generally echoed those ot their teachers in that they also 
identified the interactive and participatory elements as being a highpoint of the 
programme. Both primary and secondary pupils highlighted the ‘fun’ and 
interesting nature of the sessions and also acknowledged that this was a highly- 
effective means of learning. The following illustrate some pupils' views on the 
delivery of ln:tuition: 

I find it [role play/scenarios] easier to stay in my brain than when I’m just 
listening, It’s a bit more active so you can remember it better (primary pupil). 

It was best when we all talked together and discussed things, then did it in 
small groups, then did a role play or a poster or something (primary pupil). 

Among teachers who responded to the survey, 15 of the 22 primary teachers and 
five of the eight secondary teachers reported that pupils engaged positively with 
the activities. 


5.2.5 Views on programme content 

The fidelity information above (Tables 26 and 27) shows that teachers who did not 
deliver the entire programme were selective about which lessons they taught, 
which was likely to be based on which they felt were most relevant to the needs of 
their class. For instance, lessons on alcohol, the body and the law, on social norms, 
and on being assertive were most popular with the secondary school teachers who 
responded to the survey. 

In the majority of case-study schools, the Imtuition programme content was well 
received by staff and pupils in both primary and secondary phases, with 
interviewees generally agreeing that lessons (and associated resources) were age- 
appropriate; of high quality; and relevant to pupils’ lives; ‘it felt like they were 
learning something about life' (primary). Pupils’ interest in, and engagement with, 
the individual lessons, were seen as key to the programme’s success. 

Whilst most lessons were valued, several were identified by staff as being of 
particular benefit because of the interplay of their substantive content and the 
activities involved. For example; 

The ‘say it like it is’ session. That was a real strength of the programme - 
especially where there were mini tasks, not just whole class discussions. This 
gave less confident students the chance to participate (primary). 

In addition, the effectiveness of individual sessions related to perceived needs and 
local characteristics and contexts. In the case of the secondary schools, it was 
suggested that the alcohol and social norms sessions were especially important. 
Comments included; 

It’s a very deprived area, and alcohol is of prime importance. They ail 
wanted to know more about alcohol (secondary). 
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Social norms was important because our students do not get that from their 
parents. Understanding how to behave in certain situations is not something 
that they get from parents because the parents don’t know how to behave 
properly (secondary). 

We focussed the programme on the culture of fhe environment that we’re in. 
The society that they’re in and we’ve adapted the Intuition resources to 
meet our needs and what our children see on a daily basis (secondary) . 

Several primary school teachers noted that the content was especially useful as a 
transition tool to help prepare pupils for their move to secondary school. This point 
was also made by several pupils: - ‘we are moving onto secondary school. ..and it is 
good to do this sort of thing now’ (primary pupil). 

As well as generally enjoying the sessions, pupils suggested that the content was 
relevant and useful to them, now and in the future, equipping them with 
knowledge, awareness and strategies to deal with a range of issues and situations. 
The following comments give a flavour of how and why pupils valued the content 
of particular themes, topics and lessons. Note that although the content is different 
for primary and secondary programmes, the lesson titles and themes overlap. 


• Alcohol and the body/law 

It gives you the information for the future, so you know not to use alcohol or 
drugs (primary pupil). 

It helps us for the future, so that we don’t over-drink because it shows us the 
consequences (secondary pupil). 

• What should I do/social norms/being assertive 

It’s preparing us for the future. ...it’s probably the sort of thing that we are 
going to have to face, like bullying and stuff, and people taking drugs' 
(primary pupil). 

It was good when they gave us the scenarios and we had to say what we 
would do or not do in those situations. They were really good because they 
helped you understand about how other people feel about situations 
(secondary pupil). 

• This is me 

You had to say what you liked about yourself. It was quite nice because it 
made you feel a bit happier about yourself (primary pupil). 

• Say it like it is 

I liked doing the one where you had to break bad news to people (primary 
pupil). 
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• Seeing is believing 

I liked the one where you had to design a drink and come up with a slogan 
and a logo. It was good because we had to work as a team, all have a 
different role... It was good to work as a group, then each one had to show it 
to the rest of the class then we all discussed it. We got a lot out of that one 
(primary pupil). 


• Looking to the future 

The ones where we had to come up with our own character and say what 
they wanted to achieve when they were older. That was good because you 
could say what the challenges were that they might face (secondary pupil). 

It gave me a little example of what I could do when I’m older (primary pupil). 

5.2.6 Challenges and issues encountered in delivering the programme 

Alongside the positive response to the programme content summarised above, 
staff and pupils identified some challenges in relation to the delivery of the ln:tuition 
programme and some elements of its content. 

The main challenge for teachers was reading - and remembering - the level of 
detail in the lesson plans. Amongst those who responded to the survey, ten of the 
22 primary school teachers and four of the eight in secondary schools agreed that 
the programme required more preparation time than anticipated. Similarly, staff in 
case-study schools referred to the amount of detail contained in the lesson plans 
and the associated preparation time required for them to remember the content 
and lesson objective. For one primary school teacher, the key challenge revolved 
around; ‘getting your head round what it was asking you to do, and what it was 
asking the pupils to do’. Another commented that: ‘despite reading it the night 
before and at the dinner time of the day, I found it very difficult to remember all the 
notes so I could stand up and deliver it'. 

Some pupils commented that some sessions contained too much content and 
verbal information, and on occasion, there could be an over-reliance on 
completing worksheets: 

Sometimes it switched us off a bit when [teacher] was talking a lot. There was 
these sheets to read off and it dragged on sometimes - we thought it would 
never end (primary pupil). 

I think it was a lot to take on over 12 weeks. There was a lot of information in 
each lesson, so you remember some stuff, but some stuff you don 't remember 
(primary pupil). 

Among teachers who responded to the survey, only a minority reported facing 
technological challenges, such as access to computers (four primary teachers and 
one secondary teacher) or difficulties logging on to the programme website (three 
primary teachers and one secondary teacher). 
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However, 1 5 of the 22 primary teachers and six of the eight secondary teachers 
wanted to be able to look at the entire programme as a whole (rather than having 
to access lessons one by one on the website). Some teachers wanted paper 
materials (ten primary teachers and six in secondary schools), despite most 
agreeing that it was easy to navigate the site. Moreover, nine primary teachers and 
four secondary teachers wanted materials available in different formats (such as 
slides, or to make them accessible on mobile technology). 

Similar issues were raised by individual interviewees from the case-study schools, 
with a minority of teachers noting that they had faced some initial difficulties in 
accessing the programme on-line and that log-on details and passwords had to be 
shared between staff and pupils (in order to complete the questionnaires). One 
primary school teacher commented that ‘navigating the on-line resources was 
difficult whilst another thought that it was a shame that, because he had 
completed and signed off the programme, he was unable to access it online 
anymore. 


Suggestions for improvements 

Most of the improvements highlighted by staff in case-study schools focussed on 
addressing the difficulties they had experienced in delivering the programme. As 
such, the main suggestions included: 

• reducing the overall duration of the programme to fit into half-term schemes 
of work 

• condensing the information contained in the lesson plans to enhance clarity 

• making the lesson plans available in different formats, such as slides 

• reducing the use of worksheets and written element of the programme to 
save time and encourage greater pupil participation: 

If it's written, they are more suspicious that it might be held against them in 
the future - they are more happy to talk about it (secondary) 

• increasing the use of visual resources and interactive activities: 

There are a lot of fantastic scenarios - maybe enhance these by adding 
videos to illustrate/demonstrate them (secondary) 

• having greater differentiation of session content to increase the participation 
of students with lower levels of literacy. 

Although teachers generally liked the programme, as described above, the extent 
of the changes they suggest here provide a further indication of why many of them 
adapted the programme and its resources before delivering the content to pupils. 

Pupils were generally happy with the programme’s content and the way it was 
delivered with many interviewees not being able to suggest any specific 
improvements. However, the following suggestions for improving Imtuition were 
made by a minority of pupils: 

• address the balance between teacher-led activities and pupil activity within 
sessions, including reducing the written component of some lessons: 
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Some of the worksheets were quite boring - where you just filled them in 
without doing any activity (primary pupil). 

• provide greater opportunities for discussion to consolidate the learning from 
individual and group research activities. 

• increase IT and visual element of the programme: Could maybe have more 
videos and more things off the internet. 

• provide opportunities for pupils to be more involved in planning the lesson 
activities - / would like us to be more involved. ..to make suggestions about 
what we could do, like make it into a play or something (primary pupil). 

• modify the content of some of the sessions - some primary pupils did not 
engage with the session on future careers because it was not seen as 
relevant to them at this stage. Both primary and secondary pupils suggested 
that having opportunities to explore some session content in more detail 
would be useful, especially in relation to peer pressure and alcohol. 

You could maybe have a bit more time talking about why people have 
chosen to use substances like drugs and drink - not just why it is bad for you 
(secondary pupil). 

• introduce additional topic areas, including on-line safety and how to deal 
with abuse. 


5.2.7 The Impact of ln:tuition 

Staff and pupils in case-study schools were asked for their perceptions on the extent 
to which ln:tuition had impacted on the school, parents and pupils. These views 
should be considered in the context of the quantitative evidence described in 
Chapters 3 and 4 which found no significant evidence of impact of ln:tuition in 
primary or secondary schools. 

Overall, most case-study teachers felt that they would have achieved the same 
impact using existing provision, however one secondary school felt that because 
the programme was supplied by an external company it meant that the pupils 
were more receptive to the content than internally developed resources. Another 
secondary school said that using ln:tuition meant they had covered topics such as 
assertive behaviour earlier than they would have done otherwise. One primary 
school felt that using the Induition programme had helped the school put more 
emphasis on transition to secondary school. 


Impact on schools 

The main benefit identified by both primary and secondary school staff focussed on 
the availability of a set of high-quality resources that would save teachers 
significant amounts of lesson preparation time in the future. 

But seeing the resources, it was absolutely great - not having to think of 
something to have to do for 6 weeks was great. There was something there 
that I could manipulate and use to suit my purpose (secondary). 
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It’s definitely saved time, to have the materials all there for you, and you can 
print out what you want to use (primary). 

This seems to contradict the finding above that some teachers felt the lesson plans 
were too detailed and required unanticipated preparation time. This could mean 
views varied across interviewees, or that on one hand teachers benefited from the 
resource being readily available yet on the other hand still felt they required 
preparation time in using/adapting them (but perhaps less time than having to 
develop their own). 

In addition, it was felt that some staff, especially non-specialisfs could feel ‘out of 
their comfort zone' in delivering PHSE type curriculum, so had benefitted in terms of 
their CPD through involvement in the programme and access to Intuition's 
resources, and structured lesson plans. 


Impact on parents 

None of the interviewees noted any impact on parents arising from the delivery of 
the Intuition programme. This could be because they did not fully engage parents 
in the programme and/or did not seek their feedback. 


Impact on pupils 

Although no significant effects of the programme were evident from the 
quantitative analysis (Chapters 3 and 4), a range of perceived impacts arising from 
involvement in Intuition were identified by staff and pupils in case-study schools 
(discussed below). However, some staff questioned the extent to which 
developments were attributable solely to Intuition, and in some cases (especially in 
the primary phase) it was still too early to be able to quantify how the lessons 
learned will translate into positive change in pupils' behaviours or experience. Many 
of the perceived impacts were thought to involve slight and subtle - but 
nevertheless important - changes in pupils' attitudes and behaviour, which in many 
cases, involved preparing them for the future, rather than altering their present 
circumstances. 

Staff (especially in the secondary schools) noted that a lot of pupils lacked 
exposure to positive behaviour and culture outside of the school environment, so in 
this respect, ln:tuition was perceived to have had a very positive impact on their 
wider development and growth by showing them that there are ‘other ways' of 
acting, feeling and behaving. Staff did believe, however, that the pupils had 
become more aware of how they should behave, and why they should behave in 
that way, and also more equipped to handle themselves and difficult situations in a 
more appropriate and effective way. 

The range of impacts on pupils identified broadly fell into three main categories: 
increased knowledge and awareness; development of strategies and skills; and 
modified behaviour, all of which relate to the outcomes of interest examined by the 
trials in Chapters 3 and 4. 
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Increased knowledge and awareness 

Primary and secondary school pupils noted that through various sessions, they had 
learned new facts and increased their knowledge, especially in relation to health 
and wellbeing, most notably involving alcohol, cigarettes and other drugs. Pupils 
claimed to have developed more sophisticated understandings of the effects of 
alcohol, and its wider social implications: 

It made us understand more about how bad it can get - what too much 
drinking and smoking can do to you. Smoking and drinking can affect your 
future - like if you put it on social media, other people can see what you do 
and it might put them off giving you a job (secondary pupil). 

It’s mode me more aware of the dangers of drugs and alcohol and how it 
can really affect your life and take over your life completely. It cannot just 
damage your body, but can damage everything else like relationships and 
can drag you into bankruptcy (secondary pupil). 

I’ve learned that if you drink too much, people can take advantage of you 
and you’re in danger (secondary pupil). 


Staff and pupils also noted that the programme had enhanced their perceptions 
around alcohol in a number of ways, including raising their awareness that not all 
young people drink alcohol and that controlled drinking, in the right contexts 
(relating to age and social situation), was not necessarily a bad thing. In short, the 
sessions were said to have helped them develop a balanced view of alcohol 
consumption, and in general, most pupils contended that drinking and smoking 
was not a significant issue amongst their peers - it was generally not seen as a ‘cool’ 
thing to do: 

I know now that if it's a special occasion, like Christmas or New Year, it’s OK 
to have a couple of drinks of alcohol. But smoking is still really dangerous 
(secondary pupil). 

I know that my brother drinks too much, gets drunk every night and needs to 
sort his life out (secondary pupil). 

These comments relate to the secondary outcome about improved knowledge of 
alcohol and its effects in both the primary and secondary trials. Although staff and 
pupils reported here that knowledge had improved, which was also a finding of a 
review of fhe evidence base (Marfin et al., 2013), the statistical modelling showed 
that knowledge of pupils in the intervention group did not differ significantly from 
that of pupils in fhe control group, indicating that existing provision would have had 
the same effect. However, the analysis carried out on the knowledge outcomes 
was ITT, and as such did not take into account the extent to which a school had 
delivered ln:tuition. We know that the schools involved in the case studies had 
completed at least part of the Induition programme, and so these comments 
provide an indication that where ln:tuition had been taught, knowledge was 
perceived to have improved. Perhaps if fidelity had been better within the 
intervention group this could have translated into a discernible effect. 
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Development of strategies and skills 

Interviewees noted that ln:tuition had been effective in enhancing pupils' ability to 
cope with numerous potential social and emotional situations and challenges, with 
peer pressure being the prime focus. This was apparent for both primary and 
secondary pupils. The scenarios were seen to have been of particular use in helping 
pupils understand various situations and give them (collective) experience and 
insights into how they could respond in the future. Pupils were said to have become 
more empowered to make the right decisions, based on increased awareness and 
understanding combined with knowledge of appropriate methods and techniques. 
In general, pupils were more able than staff to highlight examples of how the 
programme had delivered positive examples, but several staff did contend that 
pupils had become more resilient and better able to make the right decisions. This 
was especially apparent in relation to primary pupils' impending transition to 
secondary school for which several teachers suggested that pupils were now they 
are well equipped to make the right decisions and keep themselves safe: 

Their awareness has definitely been raised - as a whole class, they all know 
about peer pressure and how to avoid it, but also how not to put others in a 
position where they might be pressurising others. The whole class and year 
group has benefitted in this way (primary). 


The following quotations illustrate some of the ways in which pupils felt that ln:tuition 
had helped them in terms of identifying and managing risk situations: 

In the past, we might not have thought about these things, but now we've 
done these scenarios, we now know how to react in these situations in the 
future - and what we could do to help others do in those situations 
(secondary pupil). 

It told me that it's O.k. to do what you want to do, not just to do what 
everybody else is doing and to make my own decisions (secondary pupil). 

I know now that if someone asks you to do something that you don 't want to 
do, then they’re not really your friend (primary pupil). 

Alongside enhanced decision making, ln:tuition was also seen to have facilitated 
the development of a range of other skills and social and emotional capacity 
amongst pupils. Teachers suggested that one of fhe key areas of personal 
development entailed improvements in the levels of confidence and participation 
exhibited by their pupils, underpinned by the interactive, collaborative nature of 
the programme: 

It's impacted positively on how they’ve conducted themselves in lessons, 
they work better in groups, they're more confident in expressing themselves. 
They can see more clearly what’s right and what’s wrong (secondary). 

Some pupils noted enhanced communication skills as a result of the programme - 
for themselves and others - resulting from the mechanics of the programme 
delivery as well as its substantive content. For example, the mixing of the most 
confident pupils with the more reserved class members was seen to have increased 
understanding and empathy as well as enhancing communication skills: 
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We're [four pupils] very confident and we all speak all the time. Some others 
were more shy so we encouraged us to help them. The teachers put us 
confident ones in groups with the shy people to encourage them and get 
them to open up a bit more I think if worked because they opened up more 
and got a bit more confident (primary pupil). 

Pupils and staff concurred that pupils were now better able to have difficult 
conversations with people and be assertive but not aggressive in their 
communications with others. There was a general agreement that the 
communication skills - telling people bad news was a well remembered lesson that 
will help them in their transition to secondary school. 

One lesson we all enjoyed was about someone being in a sticky situation or 
an embarrassing situation, and you had to work out how to tell them 
something in a nice way (primary pupil). 

Modified behaviour 

Given their perceived increase in knowledge and awareness around alcohol, many 
pupils claimed that their attitudes, and projected future drinking behaviour, had 
changed as a result of Induition, although this did not translate into a quantifiable 
impact within the secondary school trial (Chapter 4) . Pupils suggested very strongly 
that they would not smoke or take illegal drugs in the future, and their alcohol 
consumption would be controlled as they had more information on which to make 
their choices 

I won’t drink in the future. Not too much (secondary pupil). 

We saw a film about moderation - you don't have to drink too much to have 
a good time. You can like just drink water as well as drinking alcohol all the 
time (secondary pupil). 

Like when you go to a party, someone might bring drink. You're more aware 
of it and can choose to drink it or not (secondary pupil) . 

Like at sleepovers - there might be some drink there. But that’s up to different 
people what they choose to do. You don 't have to do it if you don 't want to 
(secondary pupil). 


5.2.8 Costs of programme delivery 

Staff in five of the case-study schools said that teaching ln:tuition meant that the 
school had incurred additional costs, over and above what they would have spent 
(two schools said no additional costs, two did not comment). The costs mentioned 
by schools were either related to teacher time (four teachers) or printing (five 
teachers), although they were not able to quantify the amount of time or expense 
incurred. As highlighted in Section 4.2.5 on challenges above, additional 
preparation time was an issue in some schools, with one of the case study primary 
school teachers commenting '/ should have put [even] more in. I didn’t realise how 
much time I would have to put in to deliver it to the children'. 
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The teachers that reported additional costs relating to printing felt that they 
needed to print a lot of the materials, although they said that where possible they 
tried to restrict the amount of printing and photocopying. One secondary school 
teacher said they were able to run the programme on screen, and therefore did 
not incur any printing costs. Responses from the teacher questionnaire indicated 
that this was perhaps less of a problem, with only five of the 22 primary school 
teachers and three of the eight secondary school teachers agreeing that the 
programme required unanticipated costs. 


5 . 2.9 Future plans for delivery of programme 

Of the 22 primary teachers who responded to the survey, just over half (12) said 
they were not sure if they would continue with the programme in the future, while 
six said they would and two said they would not (those remaining did not respond). 
Among the eight secondary school teachers, two were planning to continue with 
the programme, while one was not and three were not sure (two did not respond). 

In case-study schools, the majority of staff noted that they would like to deliver 
ln:tuition again in the next year. The key reasons for this focussed on: 

• the availability of free, quality resources reduces teacher workload 

• pupils’ interest and engagement in the programme 

• the considered value of the programme as transition tool (for primary 
schools) 

• the relevance of topic areas (and compatibility with other PHSE content) . 

However, these positive comments were accompanied by assertions that schools 
may want to make particular changes to the programme to customise it to suit the 
need and circumstances of their school. Likewise, suggestions for modification and 
adaptations included: 

• reducing the overall length of the programme and selecting the lessons most 
appropriate for particular pupil cohorts 

• using the materials as a starting point and adapting them as appropriate: 
‘dipping in is the best option.’ 


Summary 

The process evaluation found that while teachers and pupils were positive about 
the programme, schools had adapted the programme prior to delivery and a 
number of areas for improvement were suggested. Although the anecdotal 
evidence gathered through the interviews indicated that the programme had an 
impact in the areas of interest of the trial (particularly knowledge, resistance skills 
and drinking behaviour), this must be considered in the context of the quantitative 
analysis described in Chapters 3 and 4 which showed that there was no significant 
difference between pupils that had participated in the intervention and those in 
the control group. This difference could be related to the poor fidelity to the 
programme within the intervention group overall, as case study schools, all of which 
had delivered at least some of ln:tuition, reported some perceived impact. 
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Alternatively, it could also be that pupils in control schools experienced similar 
improvements as a result of their existing PSHE provision. Such improvements were 
not captured as part of the process evaluation. 
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6. Conclusions 

6.1 Limitations of the evaluation 

One threat to the internal validity of the trial came from the level of measurement 
attrition, particularly in secondary schools. Although it might be expected for 
aftrifion to be greater among control schools, it was in fact greater among 
intervention schools for both trials. Genuine programme effects on attrition could 
have led to the comparisons between the intervention and control groups being 
non-significant. This might have occurred through biased drop-out or loss of power 
masking a genuine effect of the programme on outcome measures. This possibility 
was initially investigated through the comparison of baseline characteristics for 
analysed groups (see Sections 3.2.3 and 4.2.3) and then through multiple 
imputation of the primary outcome model for the primary school trial (Section 
3.2.1 ). Insufficient data was available to undertake multiple imputation for the 
secondary trial. For the primary school trial, there was no evidence of imbalance in 
baseline characteristics for analysed groups and the non-significant effect 
remained so after multiple imputation. 

Potential bias in the findings for the secondary trial could have resulted from two 
significant differences in the baseline characteristics of pupils for whom data was 
collected at baseline and follow up. Pupils in the control group were more likely 
than intervention pupils to be an older sibling, and those in the intervention group 
were more likely than control pupils to describe themselves as belonging to a 
religious group other than Christian. Although schools (and therefore pupils) were 
randomised into groups, this difference is likely to have resulted from attrition. Note 
that both these variables were included in the primary outcome model and neither 
was significant so their imbalance perse is not a problem. However, they may be 
indicative of wider imbalance that could not be measured. 

Another limitation was the nature of the data obtained to conduct analysis on 
programme fidelity. Information was obtained from a number of sources in order to 
build a picture of the extent to which schools had delivered Induition as intended. 
One source was output from the Induition website where some users logged that 
they had delivered lessons, although data might not be complete as this was not a 
requirement (i.e. some could have delivered lessons but not logged that they had 
done so). This data was supplemented by data from the teacher questionnaire, but 
not all teachers responded. Therefore, on-treatment analysis, which took into 
consideration the number of lessons delivered and whether the most relevant 
lesson to the primary outcome had been delivered, was not necessarily complete. 
The data collected suggested that only 15 of the 34 primary intervention schools 
and five of the 21 secondary intervention schools delivered some of the 
intervention. This could explain why there were no statistically significant differences 
in outcomes between the intervention and control groups in both trials. However, 
on-treatment analysis was carried out to explore the impact of the level of fidelity 
on the primary outcomes and results remained the same. Note that the review of 
literature undertaken by Martin et al. (2013) highlighted that difficulty in judging 
impact of school-based alcohol misuse prevention programmes was partly due to 
issues related to gauging programme fidelity. 
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Feedback on why some schools (despite agreeing to participate) did not register 
for the ln:tuition site, or go on to deliver lessons after registering, was limited. One 
possible reason could have been the delay in availability of programme materials 
until the September of the autumn term. Materials were initially scheduled for 
availability in July 2013; this was communicated to schools in their initial invitation 
letter. Their delayed availability may have impacted on the ability of schools to 
timetable in lessons for the new academic year and could have detracted from 
the engagement of schools with the requirements of the trial. 

Note that the trial moved from a planned ‘effectiveness’ trial, where intervention 
schools are left completely to their own devices, to an ‘efficacy’ trial where NFER 
staff reminded schools to register for the programme, which could have altered 
results. Without being reminded, schools could have been even less likely to have 
delivered the programme with high fidelity. It is not possible to say to what extent 
external help altered fidelity but it is safe to say that the trial now represents a 
slightly exaggerated picture of how well schools might deliver the programme in 
reality. 

A process evaluation was carried out as part of the trials to explore factors 
(including fidelity, views on programme content, and challenges faced in delivery) 
which could help to explain the quantitative results. Given difficulties faced with 
general recruitment to the trial, schools included in the process evaluation were 
those willing to take part (and thus are likely to be those most engaged with the 
programme). This, and the fact that only small numbers of schools and individuals 
were included in the qualitative element of the trial, means that the process 
evaluation results are illustrative and should not be generalised. 

In order to do the planned cost-effectiveness analysis there would have needed to 
have been evidence of an effect of the programme in relation to outcomes of 
interest, which was not the case. In addition, we were not able to obtain the 
necessary expenditure data from case-sfudy schools to have been able to 
conduct this analysis if an effect had been evident. For example, school staff were 
not able to quantify the amount of money spent on printing, photocopying and the 
amount of additional time over and above what they would have spent otherwise. 


6.2 Interpretation and Discussion 

Although the development of Induition was informed by evidence-based life skills 
programmes such as ‘Unplugged’, there are differences between the programmes 
and the measures of impact that mean the results are not directly comparable. For 
example, the content differs (Unplugged covers the broader issues of tobacco and 
illicit drugs) and the Unplugged programme incorporates two and a half days of 
training for teachers who will deliver it. 

The main hypothesis that the primary school trial of ln:tuition was designed to test 
was that the ln:tuition programme improves resistance skills (confidence to manage 
peer pressure) in 10 and 1 1 year-olds. We conclude that there was no evidence of 
a positive impact on this primary outcome. There was no statistically significant 
difference between the intervention and control groups. This also means, though, 
that there was no evidence of any negative impact. No significant effects of the 
programme were found for different sub groups. 
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For the secondary school trial, the main hypothesis was to test that participation in 
the programme has an impact on the proportion of pupils aged 1 2-1 3 that are 
drinking frequently. Overall, there was no significant effect on frequency of drinking. 
However, ln:tuition was found to work differentially for males and females. In the 
intervention group, males were more likely and females were less likely to be 
frequent drinkers compared to their counterparts in the control group at follow up. 
However, further research would need to be undertaken to determine if this was a 
genuine effect of the intervention. No other significant effects on sub-groups were 
found. 

Impact on other secondary outcomes was also explored for both trials and there 
was little evidence of any effect of the programme. One exception was the impact 
on knowledge among primary school pupils. The results show that pupils in the 
intervention group had, on average, better knowledge about alcohol and its 
effects than those in the control group, although this result did not meet the p=0.05 
threshold of sfafistical significance. This could be indicafive of a genuine effect of 
the intervention on increased knowledge. Evidence from the process evaluation 
also suggested a perceived impact on pupils’ knowledge. The literature review 
carried out by Martin ef al. (201 3) found most evidence of impact of other school- 
based alcohol education and life-skills programmes was on pupils' alcohol related- 
knowledge. The literature reveals variable or inconclusive evidence in relation to 
impact on young people's decision-making skills or drinking behaviour. 

Evidence from the fidelity analysis and process evaluation indicates that not all 
teachers in the intervention group delivered the programme in its entirety - indeed 
some did not deliver it at all. Of those who did deliver lessons, views were generally 
positive about the programme’s content and teaching approaches/activities 
included in lesson plans, although most teachers adapted the programme to 
reduce its length due to a lack of curriculum time to cover it all. This finding suggests 
that the recommendation resulting from the feasibility study (Barksfield and Hull, 
2012) that steps should be taken to encourage better course fidelity, particularly by 
reducing the length of the programme, still stands. 

Teachers tailored aspects of lesson delivery to the needs of their pupils. The lessons 
were thought to be of a good quality, yet too detailed and lengthy for the entire 
content to be covered. Moreover, only a minority of teachers who participated in 
the process evaluation had sent home some of the homework tasks for pupils to 
complete with their parents/carers - none had sent all of them home. Note that the 
review carried out by Martin ef al. (2013) concluded that when parents and families 
had been included in interventions, results were usually positive. Again, this suggests 
that a recommendation resulting from the feasibility study, that parental 
engagement with the ‘home learning' activities should be encouraged, still stands. 

On-treatment analysis of the primary outcomes took into account the extent of 
programme fidelity among teachers in the trials. This did not result in any change in 
findings i.e. there was no positive (or indeed negative) impact on the primary 
outcomes (resistance skills in primary schools and frequency of drinking in 
secondary schools) depending on the extent of programme delivery. This indicates 
that the ln:tuition programme is not meeting its stated aims. 

To conclude, recruitment difficulties were faced, which was followed by school 
attrition (particularly in secondary schools) which could have had an impact on 
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results. There was little evidence of positive - or indeed negative - impact of the 
programme. Pupils in schools which delivered the intervention did no better - or 
worse - than those in schools doing their normal PSHE curriculum. Evidence from the 
fidelity analysis and process evaluation indicates that not all teachers in the 
intervention group delivered the programme in its entirety - indeed some did not 
deliver it at all. Moreover, the extent to which ln:tuition was delivered in schools did 
not impact on this result. Although teachers saw some value in the individual 
elements of fhe programme, for fhe programme fo have a positive impact in its 
entirety it needs fundamental revision. 
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